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^1.  UlttMl  --  — 

During  the  past  several  years,  computer  models  have  been  developed  for  off-site 
estimation  of  controller’s  workload.  The  inputs  to  these  models  are  audio  and 
digital  data  normally  recorded  at  an  Air  Route  Traffic  Control  Center  (ARTCC). 

This  report  summarizes  the  results  of  a  study  to  validate  the  models  at  the 
Chicago  ARTCC.  For  validation  purposes,  two  additional  estimates  of  workload 
were  used. 

One  estimate  consisted  of  journeyman  controllers  observing  peer  controllers  during 
actual  operations  and  assigning  workload  ratings.  A  second  estiaiate  of  workload 
was  obtained  from  a  survey  of  controllers  based  on  past  experiences  working  the 
sectors.  During  this  time,  audio  and  digital  data  were  collected  and  then  pro¬ 
cessed  off-line  for  inputs  to  the  controller  workload  models.  The  models  were  then 
used  to  estimate  controller  workload.  A  high  degree  of  correlation  was  found  between 
the  models’  estimated  workloads  and  controllers'  estimates  of  workload.  Based  on 
this  validation  study,  it  is  concluded  that  the  models  can  be  used  to  provide  an 
index  of  controller  workload  and  as  a  prediction  of  controller  workload  as  a  function 
of  traffic.  - 
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The  validation  tests  and  analysis  described  In  this  re¬ 
port  were  performed  by  the  System  Technology  Division, 
Transportation  Systems  Center  (TSC),  U.S.  Department 
of  Transportation.  The  work  was  sponsored  by  the  ATC 
Systems  Division,  Systems  Research  and  Development  Sei — 
vice.  Federal  Aviation  Administration  (FAA).  Over  the 
past  several  years,  the  FAA  in  conjunction  with 
SRl-International  has  developed  computerized  processes 
for  estimating  and  measuring  ATC  controller  workload. 
They  are  the  Relative  Capacity  Estimating  Process 
(RECEP)  and  the  Air  Traffic  Flow  (ATF)  model.  This 
work  represents  the  final  phase  in  the  development  of 
RECEP  and  ATF  and  was  performed  for  the  purpose  of  val¬ 
idation  of  the  processes  prior  to  their  utilization  in 
FAA  studies. 

Acknowledgement  is  given  to  the  FAA  Project  Managers, 
William  Petruzel  and  George  Scott,  who  not  only  direct¬ 
ed  the  project,  but  also  provided  liaison  with  the  var¬ 
ious  FAA  facilities  involved  and  participated  in  the 
conduct  of  field  tests.  We  also  wish  to  acknowledge 
the  support  given  by  Peter  Kovallck  and  James  Moreland 
of  the  FAA  Air  Traffic  Service  in  arranoino  the  field 


The  Chicago  Center,  and  particularly  Deputy  Chief  Mi¬ 
chael  Ciancanelli,  provided  invaluable  support  in  the 
conduct  of  field  tests.  Center  facilities,  records, 
and  staff  support  were  offered  qenerously.  Five  Jour¬ 
neymen  controllers,  A1  Broholm,  Terry  Anderson,  Ron 
Gillette,  Bernie  Miller,  and  John  Vogel,  served  as 
workpace  raters  for  an  entire  week  and  gave  expert  ad¬ 
vice  in  describing  the  functions  of  an  air  traffic  con¬ 
troller. 

John  Sigvydas  of  the  Boston  Center  provided  assistance 
in  the  availability  and  operation  of  voice  recording 
equipment  while  John  McDade,  Development  Program  Branch 
of  the  National  Aviation  Facilities  Experimental  Center 
( NAFEC) ,  was  instrumental  in  editing  digital  Systems 
Analysis  Recording  (SAR)  tapes  during  the  data  reduc¬ 
tion  phase  of  the  project. 

Kentron  International  Limited  provided  the  software 
support  for  the  project.  This  included  development  of 
computer  programs  for  processing  SAR  data  and  modifica¬ 
tion  to  the  controller  workload  models.  The  key  per¬ 
sonnel  were  Dr.  John  W.  Royal  and  Herbert  N.  Landon. 
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INTRODUCTION 


1 . I  BACKGROUND 

Over  the  past  several  years  the  FAA,  sponsoring  work  by 
SRI-Internationa 1  and  utilizing  TSC  technical  direc¬ 
tion,  has  developed  fast-time  computer  models  of  con¬ 
troller  work  activities  at  Air  Route  Traffic  Control 
Centers  (ARTCC).  The  models  are  designed  to  provide 
quantitative  estimates  of  controller  workload  under 
various  system  configurations.  Thus,  the  models  can 
provide  estimates  of  improved  controller  productivity 
attributable  to  newly-developed  equipment  sets  such  as 
the  Electronic  Tabular  Display  Subsystem  (ETABS)  0r  to 
revised  traffic  control  procedures.  The  models  also 
can  be  used  to  verify  productivity  benefits  after  new 
configurations  have  been  implemented. 

Two  computer  models  have  been  developed.  The  Relative 
Capacity  Estimating  Process  (RECEP)  provides  a  static 
measurement  of  controller  workload  as  a  function  of 
traffic  flow  rate  through  individual  sectors.  The  Air 
Traffic  Flow  (ATF)  model  dynamically  simulates  traffic 
flow  along  routes  through  a  multi-sector  area  and  pro¬ 
vides  a  continuous  measure  of  controller  workload  as 
well  as  aircraft  delays,  if  any. 


Emphasis  has  been  placed  on  the  development  of  methods 
for  the  off-line  collection  and  computer  processing  of 
work  activity  and  other  data  required  as  inputs  to  the 
models.  This  allows  field  measurement  to  be  taken  with 
minimum  effort  on  Center  operations.  It  also  provides 
means  for  processing  large  amounts  of  data  rapidly, 
greatly  enhancing  the  utility  for  the  models. 

RECEP  and  ATF  models  have  been  created  for  the  Atlanta 
and  Miami  Centers  Dased  on  limited  amounts  of  data  col¬ 
lected  and  processed  manually.  A  preliminary  compari¬ 
son  has  been  made  between  standard  NAS  Stage  A  and 
ETABS  operations  at  Miami. 

1.2  OBJECTIVES 

The  FAA  has  directed  that  a  formal  validation  process 
oe  conducted  of  the  models.  Therefore  an  experiment 
was  designed  to  simultaneously  measure  controller  work¬ 
load  by  the  RECEP/ATF  process  and  by  other 
workload-measuring  techniques.  The  measurements  were 
made  at  the  Chicago  ARTOC  under  operational  conditions 
In  II  sectors.  Results  were  then  analyzed.  The  level 
of  agreement  between  RECEP/ATF  and  the  other  measures 
of  workload  Indicates  the  validity  of  the  RECEP/ATF  mo¬ 
dels.  The  purpose  of  this  report  is  to  describe  the 
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operational  tests  which  were  conducted,  the  subsequent 
analysis  which  was  performed,  and  to  establish  the  val¬ 
idity  of  the  RECEP  and  ATF  models. 

1.3  ORGANIZATION  OF  REPORT 

Section  2  contains  a  summary  description  of  the  RECEP 
and  ATF  models.  The  models  have  been  described  repeat¬ 
edly  in  previous  documents  (Refs.  !,  2).  Therefore, 
this  section  is  not  Intended  as  an  exhaustive  descrip¬ 
tion.  Emphasis  is  placed  more  on  methodology  while 
highlighting  areas  where  changes  in  the  process  have 
Deen  made  as  a  result  of  the  work  described  herein. 

Section  3  contains  a  description  of  the  operational 
tests  performed  at  the  Chicago  ARTCC  in  which  simul¬ 
taneous  measures  of  workload  were  obtained.  The  sec¬ 
tion  also  describes  the  data  reduction  methodologies 
for  the  RECEP  and  ATF  models  and  summarizes  the  results 
obtained  using  the  models. 

Section  4  contains  analyses  that  compare  RECEP  workload 
with  other  measures  of  workload.  In  addition,  the  ma¬ 
thematical  relationship  of  components  of  workload,  as 
defined  by  RECEP,  to  traffic  flow  and  other  measures  of 
workload  is  established.  Finally,  measures  which  re- 
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late  to  sector  capacity  are  evaluated. 

Section  5  is  a  summary  of  the  conclusions  which  result 
from  the  validation  experiment. 

one  appendix  is  included.  This  appendix  contains  a 
description  of  a  controller  survey  which  was  conducted 
to  provide  various  subjective  estimates  of  workload  for 
comparison  with  RECEP  workload. 


2.  DESCRIPTION  OF  RELATIVE  CAPACITY  ESTIMATING  PROCESS 
(RECEP)  AND  AIR  TRAFFIC  FLOW  (ATF)  MODELS 

2.1  RECEP  DESCRIPTION 

RECEP  is  a  procedure  for  estimating  the  workload  of  a 
controller  or  team  of  controllers  at  an  Air  Route 
Traffic  Control  Center.  The  procedure  is  applied  on  a 
sector-by-sector  basis  and  can  be  used  to  estimate  the 
workload  of  an  individual  control  position  such  as  the 
radar  (R)  or  the  manual  (D)  or  combinations  of  control 
positions  within  a  sector.  RECEP  is  intended  to 
measure  the  primary  physical  and  mental  activities 
which  a  controller  performs,  and  results  in  a 
quantitative  estimate  of  man-minutes  of  work  performed 
during  a  specific  time  interval.  RECEP  divides 
controller  workload  into  three  basic  categories: 
routine,  surveillance,  and  conflict  prevention 
workload.  Total  workload  is  the  sum  of  the  three.  The 
work  activities  within  each  category  and  the 
measurement  and  computation  methods  are  discussed  in 
the  following  sections. 

2.1.1  Routine  Workload 

Routine  workload  consists  of  the  activities 
associated  with  air/ground  and  interphone  voice 
communications,  keyboard  operations,  and  flight 
strip  processing.  The  RECEP  procedure  involves 
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measurement  of  the  frequency  with  which  each 
activity  is  performed  within  a  specified  time 
interval.  This  frequency  is  multiplied  by  the 
average  time  a  controller  takes  to  perform  the 
activity.  Total  routine  workload  for  the  time 
interval  is  the  sum  of  the  frequency-time 
products  for  all  activities.  The  frequency  of 
occurrence  of  an  activity  will  vary  from  one 
time  interval  to  another  and  must  be  measured 
for  each  interval.  Average  performance  time  for 
each  activity  is  considered  to  be  invariant  from 
sector  to  sector  and  need  not  be  measured 
repeatedly. 

Measuring  activity  frequencies  over  many 
intervals  for  several  sectors  involves  a  large 
quantity  of  data.  Previously  it  was  necessary 
to  obtain  frequency  counts  by  observing 
controller  activities  and  manually  tabulating 
counts  in  real-time.  However,  TSC  has  developed 
techniques  by  which  all  required  data  can  be 
obtained  from  SAR  and  voice  communication  tapes 
which  record  continuously  at  all  Centers.  This 
not  only  eliminates  the  need  to  have  many 
observers  in  the  operations  area,  but  also 
allows  selection  of  sample  intervals  after 
traffic  levels  and  other  operational  conditions 
have  been  evaluated. 
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For  air/ground  and  interphone  communications, 

the  individual  activities  correspond  to  the  type 

o£  message  being  transmitted  or  received  by  the 

controller.  Bach  type  of  message  is  unique  and 

has  a  specific  average  performance  time.  The 

RECEP  definition  of  message  types  is  adapted 

from  a  method  for  categorizing  and  coding  voice 

messages  which  has  been  developed  at  NAFEC. 

This  work  is  summarized  in  Ref.  3.  The  NAFEC 

coding  system  identifies  25  major  types  of 

air/ground  messages,  with  a  more  detailed 

breakdown  within  each  type.  The  more  detailed 

breakdown  was  adopted  for  RECEP,  in  that 

messages  were  categorized  not  only  by  general 

functional  type,  but  also  with  regard  to  whether 

the  message  conveyed  information  (e.g., 

clearances,  advisories,  and  instructions) ,  asked 

a  question,  restated  previously  communicated 

information,  or  acknowledged  the  receipt  of 

information.  These  distinctions  were  made  both 

for  air/ground  and  interphone  commmunications 

(NAFEC  recently  began  applying  the  air/ground 

codes  to  interphone  messages,  although  no 

published  results  are  available  at  present). 

Another  distinction  made  just  for  air/ground 

communications  is  whether  the  message  was 

transmitted  by  the  R  controller  (controller 
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speaking)  or  by  a  pilot  (controller  listening). 
For  interphone  communications,  it  was  not 
possible  to  reliably  discriminate  whether  a 
message  was  spoken  by  the  controller  in  the 
studied  sector  or  in  another  sector.  In 
general,  the  frequency  counts  derived  for  RECEP 
are  only  for  communications  that  involved  the 
exchange  of  information.  Because  information 
exchange  requires  some  amount  of  mental  work, 
the  frequency  counts  provide  an  index  of 
workload.  Brief  acknowledgements,  such  as 
"Roger,”  and  "Wilco,"  and  salutary  messages, 
such  as  "Have  a  good  day,”  were  thus  excluded. 

Average  performance  times  for  each  air/ground 

message  type  have  been  obtained  from  a  large 

volume  of  communications  data  collected  at  the 

New  York  Common  IFR  Room  by  Princeton  University 

under  FAA  sponsorship  and  contained  in  Ref.  4. 

These  values  are  in  reasonable  agreement  with 

air/ground  message  durations  measured  at  the  Los 

Angeles  Center  by  SRI-International  (Ref.  2)  on 

a  smaller  volume  of  data.  A  large  data  bank  for 

interphone  performance  times  does  not  exist. 

For  RECEP,  a  nominal  value  of  five  seconds  is 

used  for  all  interphone  message  types.  This 

value  is  in  close  agreement  with  the  message 

durations  measured  by  SRI-International  at  Los 
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Angeles  and  with  the  mean  message  duration 
obtained  through  an  analysis  of  24  hours  of 
interphone  communications  from  the  Chicago 
Center  tests  described  herein. 

Table  2-1  is  a  listing  of  the  25  message  types 
including  average  performance  times  both  for  the 
controller  transmitting  and  receiving. 

Keyboard  operations  are  performed  by  the 
controller  for  the  purpose  of  computer  data 
entry.  As  in  the  case  of  voice  communications, 
individual  activities  are  defined  which 
correspond  to  the  type  of  message  being  entered 
into  the  computer.  Keyboard  entries  for  all 
control  positions  are  recorded  on  the  SAR  tapes. 
TSC  has  developed  procedures  to  extract  this 
data  from  SAR  and  to  code  and  tabulate  message 
frequencies  by  type.  A  total  of  19  unique 
message  types  are  identified  plus  an  "all  other" 
category.  Performance  times  for  each  message 
type  are  taken  from  the  stopwatch  measurements 
done  by  SRI-International  at  the  Los  Angeles 
Center.  Although  there  is  not  an  exact 
one-to-one  correspondence  between  message  types 
as  defined  by  SRI-International  and  by  TSC  in 
the  current  RECEP  model,  there  is  enough 
similarity  to  assure  a  high  degree  of  validity 


TABLE  2-1.  AIR/GROUND  MESSAGE  TYPES  AND  PERFORMANCE  TIMES 


MESSAGE  TYPE 

PERFORMANCE  TIME  (SECONDS) 

TRANSMITTING  • 

RECEIVING 

A/C  Vectoring/Heading 

2.9 

2.4 

A/C  Holding 

4.6 

2.4 

Altitude  Control 

3.0 

2.9 

Speed  Control 

2.9 

2.3 

Clearance  w/o  Holding 

3.1 

2.7 

Clearance  with  Holding 

4.2 

4.8 

Clearance  Delivery  -  Air  Filea 

4.6 

4.8 

Clearance  Delivery  -  Flight  Plans 

4.6 

4.8 

Call-Up 

1.6 

2.1 

Beacon  (Nondiscrete)  Control 

2.9 

2.0 

Handoff /Frequency  Change 

3.7 

2.2 

Beacon  (Discrete)  Code 

3.3 

2.3 

Mode  C  Altitude  Report 

2.9 

2.0 

A/C  Position  Report 

3.0 

2.6 

A/C  Altitude  Report 

2.0 

2.6 

Heading  and  Speed  Report 

2.3 

2.5 

Aircraft  Identification 

2.3 

1.8 

Facility  Report 

3.4 

3.0 

A/C  Traffic  Advisory 

4.9 

2.4 

A/C  Status 

2.6 

1.9 

General  Heather 

4.4 

2.3 

Alrport/Facillty  Status 

4.3 

2.3 

Specific  Weather/Flight  Conditions 

2.7 

3.7 

Altimeter  Setting 

3.6 

2.0 

Approach/Departure  Information 

3.9 

2.4 

in  the  values  assigned.  Table  2-2  lists  the  20 
message  types  and  associated  performance  times. 

Flight  strip  activities  comprise  the  remaining 
portion  of  routine  workload.  The  activities 
consist  primarily  of  marking  the  flight  strips 
to  record  the  occurrence  of  a  specific  control 
event.  One  exception  to  this  is  the  preparation 
of  a  new  flight  strip,  usually  by  the  D 
controller,  in  response  to  an  aircraft 
requesting  IFR  status  after  becoming  airborne 
(pop-up  aircraft) .  Thirteen  activities  have 
been  identified,  each  having  a  specific 
performance  time.  Again,  the  performance  times 
are  taken  from  the  SRI -International 
measurements  from  the  Los  Angeles  Center. 
Obtaining  frequency  counts  for  flight  strip 
activities  is  the  only  case  in  estimating 
workload  where  the  data  cannot  be  extracted 
directly  from  sak  or  voice  tapes.  However, 
flight  strip  markings  are  made  in  response  to 
specific  control  events  that  are  recorded  either 
by  particular  voice  messages,  keyboard 
operations,  or  SAR  traffic  counts.  Therefore, 
flight  strip  frequency  counts  are  obtained  by 
inference  from  these  other  sources.  Table  2-3 
lists  the  13  flight  strip  activities,  the 
performance  time  associated  with  each,  and  the 


TABLE  2-2.  KEYBOARD  MESSAGE  TYPES  AND  PERFORMANCE  TIMES 


MESSAGE 

PERFORMANCE  TIME  (SECONDS) 

Handoff  Acceptance 

2.0 

Handoff  Initiation  -  Manual 

3.0 

Fit  Data  Altitude  Insert 

3.0 

Fit  Data  Altitude  Amendment 

3.0 

Fit  Data  Code  Update 

3.0 

Fit  Data  Route  Amendment 

10.0 

Printout-Data  Block  Suppression 

3.0 

Pointout  Initiation 

3.0 

Data  Block/Leader  Offset 

2.0 

Data  Block  Forcing  Removal 

3.0 

Altitude  Limits  Change 

2.0 

Flight  Plan/Track  Removal 

2.0 

Flight  Plan  Readout 

3.0 

Track/Route  Display 

3.0 

Fit  Data  Update 

3.0 

Wind/Weather  Request 

3.0 

Fit  Strip  Request 

3.0 

Miscellaneous  Amendments 

3.0 

Track  Initiation 

3.0 

Other  Messages 

3.0 
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TABLE  2-3.  FLIGHT  STRIP  ACTIVITIES,  PERFORMANCE  TIMES,  AND 
FREQUENCY  COUNT  SOURCES 


ACTIVITY 

PERFORMANCE 
TIME  (SECONDS) 

FREQUENCY  COUNT 
SOURCE 

Prep  New  Fit  Strip 

10 

* 

Offline  Estimate 

Handoff  Initiation/Freq 

Change  Instruction 

2 

Controller  Air/Ground 

Handoff  Acceptance/ 

Pilot  Call-In 

2 

SAR  Traffic  Count 

Fit  Data  Altitude  Insert 

2 

Keyboard 

Fit  Data  Code  Amend 

2 

Keyboard 

Vector/Heading  Control 

2 

Controller  Air/Ground 

Altitude  Control 

2 

Controller  Air/Ground 

Speed  Control 

2 

Controller  Air/Ground 

Clearance/ Air  Filed 

2 

Controller  Air/Ground 

Clearance/Grd  Filed 

2 

Interphone 

Altitude  Report 

2 

Pilot  Air/Ground 

Heading/Speed  Report 

2 

Pilot  Air/Ground 

Altimeter  Set  Instruction 

1 

Controller  Air/Ground 

ft 

The  frequency  of  new  flight  strip  preparations  Is  based  on  average  values 
obtained  by  direct  observation  of  controller  activities  at  the  Los  Angeles 
Center.  It  Is  assumed  that  the  D-Controller  performs  this  task. 

source  from  which  the  frequency  count  is 
inferred . 

2.1.2  Surveillance  Workload 

Surveillance  workload  is  the  process  of  scanning 
the  PVD  to  retain  a  mental  picture  of  the 
traffic  situation.  SRI-International ,  in  tests 
at  the  Los  Angeles  Center,  conducted  controller 
interviews  using  video  tape  playbacks  of  actual 
traffic  situations  to  determine  the  amount  of 
time  spent  on  this  task.  On  an  average,  the 
controller  is  likely  to  look  at  an  aircraft's 
data  display  once  every  minute  with  a  dwell  time 
of  between  1.0  and  1.5  seconds  per  aircraft. 
Therefore,  a  workload  value  of  1.25  seconds  per 
aircraft-minute  is  used.  This  value  is  applied 
to  the  total  time  an  aircraft  full  data  block 
symbol  appears  on  the  PVD,  and  to  all  aircraft 
being  observed — not  just  aircraft  under  the 
sector  jurisdiction — on  the  assumption  that  a 
controller  displays  aircraft  symbols  only  if 
they  are  of  concern.  SAR  data  provides  an  exact 
count  of  the  number  of  minutes  each  symbol  is 
displayed  during  a  specific  time  interval. 
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2.1.3  Conflict  Prevention  Workload 

Conflict  prevention  workload  represents  the  time 
spent  in  detecting  potential  conflicts 
(violation  of  minimum  separation) ,  assessing  the 
situation,  and  taking  corrective  action.  In 
general,  two  types  of  potential  conflicts  can 
occur:  crossing  conflicts  where  the  projected 
flight  paths  of  two  aircraft  intersect  with  less 
than  minimum  altitude  separation,  and  overtaking 
conflicts  where  aircraft  are  on  the  same  flight 
path  at  different  speeds. 


General  equations  for  estimating  the  expected 
number  of  conflicts  per  specific  time  period 
have  been  derived  by  Siddiqee  (Ref.  5)  and 
Dunlay  (Ref.  6).  The  equation  used  in  this 
study  for  the  expected  numbers  of  crossing 
conflicts  per  unit  time  period  at  the 
intersection  of  two  flight  paths  is: 


C  = 


2  fl  f2  * 


R 


v. 


-  2  vi  v2  COS  ° 


(1J 


Vx  V2  sin  a 


where 


C  is  the  expected  number  of  conflicts  per 
unit  time  period; 


f1  and  f2  are  the  flow  of  aircraft  along 
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flight  paths  1  and  2  respectively  (aircraft  per 
time  per iod) ; 

V.^  and  V2  are  the  average  ground  speeds  of 
aircraft  along  flight  paths  1  and  2  respectively 
(nautical  miles  per  time  period); 

a  is  the  angle  of  intersection  between  the 
two  flight  paths; 


X  is  the  separation  minimum  (nautical 
miles)  . 


The  equation  used  to  determine  the  expected 

number  of  overtakes  along  a  flight  path  is: 
n-1  n 


i=l 

where 


(Sl+x)fi 


vi 


k=i+l 


(Vi  -  vk) 


(2) 


0  is  the  expected  number  of  overtakes  per 
unit  time  per.  id; 

n  is  the  number  of  discrete  speed  classes 
along  the  route; 

f^  and  fk  are  the  flow  of  aircraft  at  the 
ith  and  kth  speed  classes  respectively  (nautical 
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miles  per  time  period); 


and  Vk  are  the  average  ground  speeds  of 
the  ith  and  kth  speed  classes  respectively 
(nautical  miles  per  time  period); 

X  is  the  separation  minimum  (nautical 
miles)  . 

The  crossing  conflict  equation  was  derived  on 
the  assumption  that  the  flight  paths  were  both 
level.  In  practice,  the  flight  path  angles  of 
transitional  aircraft  in  an  enroute  sector  are 
small.  Therefore,  the  above  conflict  equation 
can  be  used  for  transitional  as  well  as  level 
flight  paths. 

Potential  conflicts  exist  when  two  or  more 
flight  paths  merge  into  one.  It  can  be  shown 
that  the  crossing  conflict  equation  is  an 
excellent  approximation  for  the  expected  number 
of  conflicts  due  to  the  merging  of  flight  paths. 

The  above  approx iaations  are  in  agreement  with 
previous  work.  1.  Appendix  D  of  Ref.  1,  S.R.I. 
states  that  the  level-level  crossing  equation 
should  be  used  for  the  above  two  cases. 
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Another  type  of  potential  conflict  is  the 
altitude  intersection  of  two  flight  paths  along 
the  same  ground  track.  A  form  of  the  crossing 
conflict  equation  could  be  applied  to  this  case 
(Ref.  1).  However,  since  the  angle  between  the 
two  flight  paths  is  small,  the  overtake  equation 
will  be  used.  This  is  in  agreement  with  S.R.I. 
Atlanta  case  study  (Ref.  7). 

The  last  type  of  potential  conflict  analyzed  was 
the  case  of  two  aircraft  heading  toward  each 
other.  In  this  case,  the  expected  number  of 
potential  conflicts  was  considered  1.0. 

Utilizing  the  above  equations  and  assumptions, 
the  method  of  computing  conflict  prevention 
workload  for  a  specific  time  interval  is 
discussed  in  the  following  paragraphs. 

For  each  sector,  many  samples  of  traffic  flow 
are  obtained  from  SAR  data.  A  fifteen-minute 
sample  interval  is  used  because  this  value 
approximates  the  average  sector  flight  time. 
Therefore,  the  number  of  aircraft  entering  the 
sector  can  be  directly  equated  to  flow  rate. 
The  flight  path  of  each  aircraft  is 
reconstructed  from  SAR  data  and  the  paths  are 
analyzed  for  potential  conflict  points.  The 
probability  of  aircraft  actually  being  in 
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conflict  is  then  computed  based  on  previously 
delineated  criteria.  A  workload  value  to 
resolve  conflicts  is  obtained  by  multiplying  the 
number  of  probable  conflicts  by  an  average  time 
to  resolve  conflicts.  A  value  of  60  seconds  and 
40  seconds  is  used  for  crossing  conflicts  and 
overtaking  conflicts,  respectively  (Ref.  1). 
Thus  for  each  sample,  a  data  point  is  obtained 
which  relates  conflict  workload  to  the  number  of 
aircraft.  Based  on  many  sample  data  points,  a 
function  defining  workload  versus  aircraft  is 
obtained.  This  function  typically  takes  the 
form  of  a  quadratic: 


W8 

*  cnJ 

where  W8 

*  conflict  prevention  workload 

expressed  in  man-minutes  per 

sample  interval 

C 

*  a  constant  in  units  of 

man-minutes  per  sample 

2 

interval  per  (aircraft) 

N« 

*  number  of  aircraft 

The  slope  of  the  quadratic  function  (C)  derived 


from  sampled  data  is  then  used  to  compute 
workload  for  any  specified  time  interval: 


wc  *  PI  CN2  (4) 

PC 

where  Wc  =  conflict  prevention  workload 
expressed  in  man-minutes 
per  time  interval 


Pj  =  duration  of  time  interval  in 
minutes 


P  *  duration  of  sample  interval 

u 

in  minutes 


N  =  mean  number  of  aircraft  under 
control 


An  example  of  conflict  prevention  workload 
calculations  is  contained  in  Section  3.6. 


2.2  ATF  DESCRIPTION 

ATP  is  a  computerized  fast-time  simulation  of  aircraft 
flow  along  defined  routes  within  a  multisector  area  of 
an  ARTCC .  ATF  can  be  used  to  simulate  traffic  flow 
within  an  individual  sector  or  an  entire  Center. 
Typically,  an  area  of  from  10  to  12  contiguous  sectors 
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is  simulated.  An  empirical  traffic  sample,  in  the 
order  of  six  to  eight  hours,  is  used  to  define  a  route 
structure  and  traffic  flow  along  each  route  for  the 
entire  area.  Routes  are  divided  into  arcs  which 
correspond  to  the  segment  of  the  route  traversing  an 
individual  sector.  Aircraft  are  sequenced  along  a 
route  from  sector  to  sector  based  on  average  arc 
transit  times.  Sequencing  is  performed  minute  by 
minute . 

Workload  is  computed  for  each  sector  minute  by  minute 
using  the  following  relationship  (Ref.  1): 


where 


K 


1 


K 


2 


K  ,N  +  K„N2 
1  2 

workload  in  man-seconds 
per  computation  interval 
coefficient  in  man-seconds 
per  computation  interval 
per  aircraft 

coefficient  in  man-seconds 
per  computation  interval 
per  (aircraft)2 
number  of  aircraft  in  sector 


(5) 


Coefficients  are  determined  by  measuring  total  RECEP 
workload  over  many  sample  intervals  and,  by 
curve-fitting,  deriving  a  second  order  function  of 
workload  versus  aircraft.  The  form  of  the  function 
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assumes  that  routine  and  surveillance  workload  vary 
linearly  with  aircraft,  that  conflict  workload  varies 
as  the  square  of  aircraft,  and  that  there  is  zero 
workload  with  zero  traffic. 

A  workload  limit  is  assigned  to  each  sector.  ATP  has 
look-ahead  capability  to  sense  when  saturation  is  about 
to  occur.  Under  this  condition,  aircraft  are  delayed 
from  entering  the  sector  until  the  saturation  is 
relieved.  Traffic  can  be  artifically  increased  in 
order  to  investigate  saturation  conditions  within  the 
area. 

Output  measures  of  the  ATF  model  are  workload  for  each 
sector  and  aircraft  flow  rate  and  delays  either  by 
route  or  sector.  Output  measures  can  be  summed  or 
averaged  for  specific  time  intervals. 

The  primary  uses  of  the  ATP  model  are  in  measuring 
system  delay  characteristics  under  alternate  system 
configurations  and  in  evaluating  sector  workload  over 
long  periods  of  time  and  varying  traffic  conditions. 

2.3  SUMMARY  OF  CHANGES  TO  RECEP/ATF 

Over  the  past  several  years,  SRI-Znternational ,  under 
FAA  sponsorship,  has  developed  the  RECEP/ATF  process 
and  applied  it  in  several  FAA  studies.  Numerous 
descriptive  reports  have  been  generated  and  have 
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received  wide  circulation.  In  conducting  the  Chicago 
ARTCC  validation  tests  described  herein,  TSC  has  made 
refinements  to  the  process.  This  section  contains  a 
brief  review  of  those  refinements  for  the  purpose  of 
calling  attention  to  those  areas  where  previous 
concepts,  perhaps  firmly  established,  do  not  apply. 

2.3.1  Definition  of  Routine  Workload  Activities 

In  the  past,  the  subdivision  of  routine  workload 
into  individual  activities  was  done  on  a 
functional  basis.  There  was  no  requirement  that 
the  activities  correspond  to  a  previously 
defined  coding  system.  TSC,  however,  in 
developing  automated  SAR  data  extraction 
programs  and  in  adopting  the  NAFEC  voice  coding 
system  was  forced  to  deviate  slightly  from 
previous  definitions  of  activities.  The  sum 
total  of  all  activities  still  accounts  for  total 
routine  workload,  however.  In  addition, 
previous  RECEP  models  have  included  as  a 
component  of  routine  workload  the  time  spent  in 
direct  face-to-face  conversations  between 
controllers.  TSC  has  eliminated  direct  voice  as 
a  workload  component  on  the  basis  that  these 
conversations  are  for  the  purpose  of 
verification  and  coordination  and  do  not  add 
fundamentally  to  the  workload. 


23 


2.3.2  Activity  Per formance  Times 


Activity  performance  times  were  previously 
defined  by  SRI-lnternational  in  terms  of  the 
minimum  time  required  to  perform  an  activity. 
In  the  present  RECEP  model,  average  times  based 
on  a  large  sample  of  data  are  used  for  voice 
communications  activities,  but  minimum  times 
obtained  from  SRI-lnternational  reports  are  used 
for  the  other  routine  activities,  namely, 
keyboard  entry  actions  and  flight  strip 
operations.  It  is  suspected  that  average 
activity  durations  more  accurately  reflect 
differences  between  control  activities  than 
minimum  durations  do.  In  the  large  sample  of 
voice  communications  data  (Ref.  4),  the  minimum 
durations  of  many  different  message  types  are 
approximately  equal,  all  on  the  order  of  one 
second;  however,  the  average  durations  for 
those  different  message  types  differ.  It  is 
noteworthy  that  the  average  durations  of  voice 
messages  in  those  large  samples  are  on  the  same 
order  of  magnitude  as  the  minimum  times  that  are 
reported  or  evident  (through  subdivision  of 
voice  communications  transactions  into  their 
component  messages)  in  the  work  by 
SRI-lnternational.  The  agreement  is  probably 
due  to  the  fact  that  SRI's  minima  were  drawn 


from  relatively  small  samples  of  data. 
Increasing  the  sample  size  would  have  increased 
the  opportunity  for  observing  the  very  short 
(low  probability)  minimum  durations  that  are 
actually  seen  in  large  samples.  The  minimum 
times  reported  by  SRI-International  for  the 
other  routine  activities,  keyboard  entries  and 
strip  operations,  might  also  prove  to  be 
approximately  equal  to  the  average  durations  in 
large  samples  of  activity  data,  which  are  now 
unavailable.  Accordingly,  the  working 

hypothesis  is  that  all  the  performance  times 
used  for  the  present  version  of  RECEP — both  the 
large  sample  averages  for  voice  communications, 
and  the  small  sample  minima  for  other 
activities — are  average  durations. 

2.3.3  Surveillance  Workload 

In  previous  versions  of  RECEP,  the  R 

controller's  surveillance  workload  was  based  on 
the  average  number  of  minutes  that  all  aircraft 
were  under  the  jurisdiction  of  a  sector.  This 
average  sector  flight  time  was  used  to  estimate 
the  average  surveillance  workload.  This 

approach  has  two  difficulties.  First,  R 
controllers  often  monitor  aircraft  that  are  not 

under  their  jurisdiction.  The  airspace  is  a 
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continuous  volume,  and  controllers  are  alert  to 
traffic  near  the  borders  of  the  airspace  over 
which  they  have  jurisdiction.  Secondly, 
surveillance  workload  varies  from  one  time 
interval  to  another.  This  variability  should  be 
measured  when  comparing  RECEP  workload  to 
workpace  ratings  taken  every  five  minutes. 
Since  the  ratings  vary  over  time,  the  measure  of 
RECEP  workload  should  be  capable  of  analogous 
variations.  Whether  the  two  measures  of 
workload  vary  in  time  together  to  an  appreciable 
degree  is  one  of  the  questions  that  bears  on  the 
validity  of  RECEP  workload  measures. 

Therefore,  in  the  present  version  of  RECEP,  the 
R  controller's  surveillance  workload  is  based  on 
the  average  number  of  aircraft  full  data  blocks 
that  are  displayed  at  the  same  time  on  the 
sector's  Plan  View  Display.  These  data  blocks 
represent  aircraft  that  are  under  the  sector's 
jurisdiction,  pointed  out  to  the  sector  by 
controllers  in  another  sector,  or  selected  for 
display  (i.e.,  "forced")  by  the  controllers  in 
the  sector.  The  number  of  aircraft 
simultaneously  displayed  is  sampled  once  every 
five  minutes,  and  the  average  number  is 
calculated  for  three  successive  five-minute 
intervals  to  give  the  15-minute  value.  This 
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sampling  procedure  is  analogous  to  the  one  used 
for  workpace  ratings,  that  is,  a  rating  once 
every  five  minutes,  and  15-minute  averages  based 
on  three  successive  ratings. 

.3.4  Sector  Capacity/Wor kload  Limit 

Past  RECEP  models  included  the  concept  that  each 
sector  is  limited  in  traffic  capacity,  expressed 
in  aircraft  per  hour,  which  corresponds  to  an 
upper  limit  in  the  workload  a  controller  can 
perform.  Values  of  workload  limit  were  found  to 
be  48  man-minutes  per  hour  for  a  Radar 
Controller  and  66  man-minutes  per  hour  for  a 
combined  Radar/Manual  Controller  team  in  several 
sectors  evaluated  at  the  Los  Angeles  Center. 
Several  aspects  of  this  concept  are  now  being 
questioned  as  a  result  of  the  work  performed  at 
the  Chicago  Center.  First,  controllers 
apparently  do  not  think  of  traffic  capacity  or 
workload  in  hourly  terms.  Aircraft 
simultaneously  under  control  is  more  meaningful 
and  would  serve  as  a  better  definition  of 
capacity  conditions.  Secondly,  during  the 
Chicago  tests,  several  sectors  were  operating  at 
capacity  conditions  (as  defined  by  "very  heavy” 
workpace  ratings)  for  periods  of  five  minutes  or 
more.  The  RECEP  workload  values  for  these 
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periods,  however,  varied  from  sector  to  sector. 
It  is  therefore  questionable  that  one  workload 
value  such  as  48  man-minutes  per  hour  can  be 
used  to  define  a  limit  for  all  sectors. 
Finally,  the  absolute  values  of  48  and  66 
man-minutes  per  hour  appear  to  be  low.  During 
the  tests  at  the  Chicago  Center  described 
herein,  there  were  numerous  incidents  of 
workload  as  computed  by  RECEP  exceeding  15 
man-minutes  during  a  15-minute  time  interval. 
Admittedly,  the  tests  were  conducted  at  the 
busiest  Center  in  the  country  and  only  busy 
hours  were  selected.  The  fact  is,  however,  that 
RECEP  measures  the  time  spent  on  activities 
which  may  be  performed  simultaneously,  thus 
making  it  possible  to  exceed  100%  of  the  total 
time  available.  Expressing  workload  in  units  of 
man-minutes  is  more  a  measure  of  the  busyness  of 
a  controller  rather  than  an  absolute  measure  of 
working  time  versus  idle  time. 

2.3.5  Sample  Interval  Duration 

In  the  past,  RECEP  workload  values  and  ATF 
coefficients  were  computed  from  data  combined 
over  one-hour  observation  intervals.  A  shorter 
sample  provides  more  accurate  workload  data  for 
several  reasons.  First,  more  frequent  sampling 
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gives  a  truer  indication  of  workload  variation 
over  time.  This  is  the  reason  why  workpace 
ratings  are  ordinarily  made  once  every  5  or  10 
minutes  during  FAA  studies.  Second,  in  the  case 
of  conflict  prevention  workload,  sampling  on  an 
hourly  basis  in  order  to  compute  conflict 
probability  results  in  an  erroneously  high 
value.  The  over estimation  occurs  because 
aircraft  are  treated  as  if  they  are  in  the 
sector  at  the  same  time,  when  the  aircraft  are 
actually  separated  in  time  by  as  much  as  three 
sector  flight  times,  and  therefore  cannot 
interact  with  each  other.  A  sample  interval  of 
15  minutes  has  been  selected  for  the  present 
work  because  it  approximates  the  average  sector 
flight  time  over  all  sectors,  has  a  greater 
capability  than  60  minutes  for  reflecting 
workload  variations,  and  is  still  consistent 
with  practical  constraints  on  data  reduction  and 
analysis. 

2.3.6  Measurement  of  Aircraft  Flow 

In  determining  the  constant  coefficients  in  the 
ATF  equation  for  computing  workload,  total  RECEP 
workload  is  measured  for  many  sample  intervals 
and,  by  curve-fitting,  a  function  of  workload 
versus  aircraft  is  derived.  In  the  past, 
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traffic  flow  rate  (TFR)  has  Deen  used  as  the 
measure  of  the  number  of  aircraft  in  the  sample. 
TFR  is  defined  either  as  aircraft  entering  plus 
aircraft  exiting  divided  by  two  or,  simply,  as 
aircraft  entering  during  the  time  interval. 
However,  in  the  ATF  model,  workload  is 
determined  by  multiplying  the  coefficients  by 
the  number  of  aircraft  in  the  sector  uuring  each 
computation  interval.  TFR  is  not  the  same 
quantity  as  aircraft  in  the  sector  and  an 
erroneous  workload  value  results.  The  mean 
number  of  aircraft  under  control  (AUC)  during 
each  sample  interval  is  a  more  accurate  measure 
of  the  number  of  aircraft  in  the  sector  and  is 
analogous  to  the  aircraft  flow  parameter  used  in 
the  ATF  model.  The  current  RECEP  model  uses  AUC 
as  the  measure  of  aircraft  flow. 
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I  3.  VALIDATION  FIELD  TEST  AND  DATA  REDUCTION 

I 

RECEP  and  ATF  have  their  greatest  value  for  evaluating 
controller  workload  at  high  traffic  levels.  It  is  for 

L  high  traffic  levels  that  improvements  to  the  ATC  system 

have  been  designed,  in  order  to  reduce  the  workload  of 
controllers  per  aircraft,  and  so  permit  controllers  to 
safely  handle  more  aircraft.  Because  the  study  to 
validate  the  RECEP/ATF  models  should  be  conducted  in 
sectors  where  reasonably  high  traffic  levels  could  be 
expected  often,  the  Chicago  Center  was  selected.  Daily 
traffic  counts  for  the  Center  for  the  year  1977  were 
reviewed  and  from  this  a  test  period  from  2/28/78  to 
3/3/78  was  selected  where  higher  than  average  traffic 
could  be  expected.  By  choosing  the  busiest  hours 
with in  this  period,  it  was  felt  that  a  reasonable 
number  of  very  high  traffic  samples  would  be  obtained. 
The  Center  was  most  cooperative  in  scheduling  the  tests 
and  supplying  the  necessary  staff  support  and 

facilities. 

i 

i 

! 

3.1  CHICAGO  CENTER  OPERATIONAL  ASPECTS 

In  deriving  RECEP  and  ATF  models  of  controller 
workload,  certain  aspects  of  the  operational 
environment  are  of  interest.  These  are  discussed 
below. 
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The  Chicago  Center  has  approximately  40  active  sectors 
which  operate  individually  during  the  day  and  evening 
shifts.  Some  are  combined  during  the  mid-shift.  The 
sectors  are  divided  into  seven  areas  of  specialization. 
Controllers  are  qualified  to  work  any  sector  within  an 
area.  The  layout  of  the  sectors  is  centered  around 
O'Hare  Airport.  There  are  alternately  four  arrival  and 
four  departure  sectors  aligned  on  the  major  points  of 
the  compass.  Most  O'Hare  traffic  is  fed  directly  from 
or  to  eight  high  altitude  sectors  which  are  also 
aligned  radially.  There  are  approximately  20  outlying 
low  alti*-_.ue  sectors  and  three  superhigh  sectors  which 
primarily  handle  enroute  traffic. 

In  planning  an  ATF  model  of  a  portion  of  the  Chicago 
Center,  it  was  desirable  to  select  contiguous  sectors 
arranged  to  contain  continuous  flows  of  traffic. 
Eleven  sectors  west  of  O'Hare  were  chosen.  These 
include  the  West  Departure  sector  and  Farmm  and  Vains, 
the  northwest  and  southwest  arrival  sectors, 
respectively.  Also  included  are  the  four  western  high 
altitude  sectors,  Dubuque,  Iowa,  Bradford  and  Joliet; 
one  superhigh  sector,  McCook,  which  overlays  Iowa, 
Bradford  and  much  of  Joliet;  and  three  underlying  low 
altitude  sectors,  Rockford,  Moline  and  Peoria.  These 
11  sectors  account  for  30%  of  the  Center  traffic  and 
contain  all  of  the  O'Hare  arrivals  and  departures  to 
the  west.  The  areas  involved  are  West  Terminal,  West 

32 


V.*. 


High  Altitude,  and  West  Low  Altitude 


Traffic  at  the  Chicago  Center  is  quite  heavy  from  7  AM 
to  7  PM.  Peak  loads  occur  throughout  the  day. 
Pronounced  peaks  regularly  occur  at  about  9  AM,  1  PM,  3 
PM,  and  5  PM.  In  order  to  consolidate  the  hours  of 
data-taking  and  other  activities,  the  period  from  noon 
to  5  PM  was  designated  as  the  test  period.  Within  that 
period,  individual  hours  were  selected  for  data 
reduction  and  analysis  after  traffic  levels  and  other 
considerations  had  been  evaluated. 

The  controller  team  composition  within  sectors  normally 
includes  a  Radar  (R)  Controller  and  Manual  (D) 
Controller.  In  addition,  coordinators  are  assigned  to 
coordinate  traffic  flow  between  several  sectors.  As  an 
example,  one  coordinator  may  work  between  an  arrival 
sector  and  the  several  sectors  which  feed  it* 
Assistant  (A)  Controllers  prepare  and  distribute  flight 
strips  to  the  appropriate  sectors.  One  A-Controller 
normally  services  an  entire  area.  Handoff  (H) 
Controllers  are  sometimes  assigned  to  a  sector  team 
under  extremely  busy  conditions.  The  function  of  the 
H-Controller  is  to  assist  the  team  in  coordination, 
keyboard  entries,  and  interphone. 

At  the  time  of  the  validation  tests,  the  9020  computer 
software  version  in  effect  was  NAS  Stage  A3d2.4. 
Controller  workload  models  derived  during  these  tests 
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would  be  valid  for  most  software  versions  being  used  in 
the  field.  However,  if  the  system  configuration  were 
changed  significantly,  the  models  would  require 
amendment . 

3.2  CONDUCT  OF  TESTS 

The  primary  objective  of  the  validation  tests  was  to 
obtain  a  comparison  of  controller  workload  as  measured 
by  RECEP  with  other  measures  of  workload  for  the 
purpose  of  validating  the  RECEP  process.  Two  other 
measures  were  used:  workpace  ratings  and  a  subjective 
rating  of  workload  obtained  by  a  survey  of  controllers. 
The  general  approach  followed  was  to  gather  RECEP  and 
workpace  data  over  a  four-day  period  while  monitoring 
test  conditions  and  then  selecting  specific  time 
intervals  on  which  to  base  the  comparison.  All 
measures  of  workload  were  for  the  R-Controller  position 
which  has  the  heaviest  workload  and  is  the  most 
critical  in  defining  sector  capacity. 

As  stated  previously  all  data  required  for  RECEP  and 
ATF  modelling  are  gathered  continuously  offline  on  SAR 
and  voice  tapes.  However,  for  the  purpose  of  the 
validation  tests,  it  was  desirable  that  all  test 
intervals  selected  reflect  a  uniform  set  of  test 
conditions.  Therefore,  two  observers  were  present  in 
the  operations  area  during  all  test  periods  to  monitor 
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composition  was  of  primary  interest.  If  the 
R-Controller  was  assisted  by  an  H-Controller ,  these 
periods  were  rejected  because  in  reducing  taped  data  it 
was  not  possible  to  distinguish  between  the  R  and  H 
work  contributions.  In  addition,  if  the  R-Controller 
was  also  working  the  D  position,  this  was  noted  because 
additional  data  channels  would  then  be  searched  in 
computing  workload.  Periods  which  involved  unusual 
traffic  conditions  such  as  holding  patterns  or 
rerouting  of  traffic  due  to  changes  in  active  runways 
at  O'Hare  were  noted  and  rejected  as  test  intervals. 
Test  intervals  were  also  based  on  uniform  weather 
conditions  as  far  as  possible. 

Workpace  rating  is  a  technique  used  by  the  FAA  to 
estimate  workload  level.  A  peer  journeyman  controller 
observes  the  work  activities  of  a  controller  on  duty 
and  subjectively  rates  the  work  level  on  a  seven-point 
scale  ranging  from  "very  light"  to  "very  heavy." 
Ratings  are  made  at  five-minute  intervals  and  can  be 
continued  for  a  period  of  one  hour  or  more.  Table  3-1 
contains  a  list  of  various  ratings  and  a  definition  of 
each.  From  the  11  sectors  being  modeled,  five  of  the 
busiest  were  chosen  for  workpace  ratings.  They  are 
Vains,  Farmm,  Moline,  Peoria,  and  Bradford.  Volunteers 
served  as  workpace  raters  for  sectors  in  which  they 

were  qualifed,  with  two  raters  alternating  between 
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TABLE  3-1.  WORKPACE  DEFINITIONS 


o  Very  Light  Workload  (VL) .  A  "VL"  rating  should  be  assigned  when  the  Workpace 
level  is  so  low  that  relatively  little  attention  has  to  be  paid  to  the  posi¬ 
tion  of  operation.  Minimal  exertion  is  required. 

o  Light  Workload  (L).  An  "L"  rating  should  be  assigned  when  the  Workpace  is 
such  that  more  than  minimal  exertion  is  required,  but  the  complexity  of 
situations  is  such  to  only  engage  the  controller's  complete  attention 
periodically.  There  are  no  complex  control  situations. 

o  Average  Workload  (A).  An  "A"  should  be  assigned  when  the  situation  com¬ 
plexity  requires  almost  full-time  attention  of  the  controller.  The  workload 
is  evenly  distributed  and  places  no  unusual  demand  upon  the  controller. 

This  pace  could  be  maintained  up  to  an  8-hour  period  with  normal  relief. 

-  Gradient.  A-  should  be  assigned  when  significantly  less  than  full 
attentiveness  is  required  at  the  position;  the  demands  placed  upon  the 
controller  are  slightly  less  than  one  could  expect  at  average.  Infrequent 
periods  of  inactivity  occur. 

♦  Gradient.  A+  should  be  assigned  when  the  demands  are  slightly  greater 
than  A.  Rare  periods  of  inactivity,  full  attentiveness  to  the  position  is 
required.  A  controller  could  be  expected  to  work  at  this  pace  up  to  six 
hours  with  normal  relief. 

o  Heavy  Workload  (H).  An  "H"  rating  should  be  assigned  when  the  complexity 
and  exertion  required  to  cope  with  the  situation  necessitate  rapid  deci¬ 
sions;  there  is  constant  operational  activity.  Demands  placed  upon  the 
controller  exceed  those  of  a  normal  pace.  A  controller  could  be  expected 
to  securely  deal  with  this  level  of  work  for  up  to  3  hours. 

o  Very  Heavy  (VH).  A  "VH"  should’ be  assigned  when  there  is  continuous, 
laborious  activity;  superior  exertion  is  required  and  the  rapidity  of 
response  and  thinking  processes  are  critical.  There  are  delays  in 
acknowledging  demands  placed  upon  the  position.  A  controller  would  be 
"pushed"  to  maintain  this  pace  for  l  hour. 
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Vains  and  Farmm  and  two  between  Peoria  and  Holine.  A 


single  rater  made  all  observations  for  Bradford. 
Ratings  were  made  for  three  one-hour  periods  on  each  of 
the  four  days.  The  ratings  were  converted  to  a 
numerical  scale  from  1  to  7  and  ratings  for  three 
successive  five-minute  periods  were  summed  and  averaged 
to  obtain  a  mean  value  of  workpace  for  each  15-minute 
period.  From  this  total  set  of  workpace  data,  24 
15-minute  periods  per  sector  were  later  selected  as 
test  intervals  for  comparison  with  RECEP.  The 
selection  was  based  on  high  workpace  ratings  and  on 
satisfying  uniform  test  conditions.  It  should  be  noted 
that  one  day  was  spent  prior  to  the  beginning  of 
testing  familiarizing  the  raters  with  the  workpace 
technique  and  in  trial  runs. 

During  the  four-day  test  period,  off-line  interviews 
were  conducted  with  volunteer  controllers  using 
questionnaires  designed  to  estimate  workload  levels  in 
the  11  sectors.  Controllers  were  asked  to  rate  only 
those  sectors  in  their  area  of  specialization.  Five 
controllers  in  each  of  the  three  areas  participated. 
Three  questionnaire  forms  were  used.  The  first 
provided  a  listing  of  relative  sector  difficulty  within 
an  area  based  on  a  seven-point  scale  ranging  from 
"easiest  to  control"  to  "most  difficult  to  control" 
during  typical  busy  periods.  The  second  form  provided 
an  estimate  of  the  maximum  sector  traffic  capacity. 
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The  questionnaire  originally  asked  for  an  hourly 
maximum  capacity  estimate.  However,  controllers  are 
better  able  to  estimate  maximum  simultaneous  aircraft. 
Therefore,  the  latter  is  the  estimate  obtained  from  the 
interviews.  The  third  questionnaire  form  provided  a 
seven-point  workpace  estimate  versus  the  number  of 
simultaneous  aircraft  under  control,  again  ranging  from 
"very  light"  to  "very  heavy".  The  ratings  obtained 
from  these  interviews  were  used  in  the  RECEP  validation 
analysis  discussed  in  Section  4.  The  Appendix  contains 
a  description  of  the  interview  process  and  examples  of 
the  questionnaire  forms. 

3.3  RECEP/ATF  DATA  REDUCTION  METHODOLOGY 

SAR  tapes  run  continuously  collecting  digital  data  on 
Center  operations  for  all  sectors.  Because  of  the  high 
volume  of  traffic  at  Chicago,  each  tape  contains  only 
15  minutes  of  data.  SAR  tapes  for  the  test  period, 
noon  to  5  PM,  for  each  of  four  days  were  shipped  to 
Code  ARD-141 ,  NAFEC,  where  a  DART  editing  and 
cataloging  operation  was  performed  to  provide  edit 
tapes  of  a  DART  LOG  data  base.  The  edit  tapes  were 
then  shipped  to  TSC.  TSC  has  developed  FORTRAN 
computer  programs  which  extract  specific  information 
from  the  edit  tapes  required  for  RECEP  and  ATF 
modelling.  The  programs  are  run  on  an  IBM-360  machine 
which  is  compatible  in  language  and  symbology  with  the 


38 


9020  computers  used  in  the  field  and  at  NAFEC.  Table 
3-2  contains  a  listing  of  the  RECEP/ATF  parameters 
which  are  derived  from  the  SAR  data.  Where  applicable, 
the  parameters  are  segregated  by  sector  and  by  test 
interval.  The  programs  also  provide  hourly  summations 
of  appropriate  parameters. 

Voice  tapes  also  run  continuously  recording  air/ground 
and  interphone  communications  for  all  control  positions 
within  all  sectors.  Each  tape  contains  16  hours  of 
communication.  Voice  tapes  for  the  four  days  of 
testing  were  taken  to  the  Boston  Center  where  playback 
tapes  were  made  for  the  test  intervals  selected  for 
each  of  the  11  test  sectors.  A  typewritten 
transcription  was  made  from  the  playback  tapes.  Each 
message  was  then  coded  in  accordance  with  the  list 
contained  in  Table  2-1  and  frequency  counts  of  message 
types  were  tabulated  for  each  test  interval. 

Voice  messages  for  the  Radar  position  in  the  Farmm 
sector  were  not  obtained  due  to  a  malfunction  in  the 
recorder  at  the  Center.  As  a  result,  routine  workload 
could  not  be  estimated  based  on  direct  measurements. 
This  problem  was  circumvented  by  computing  the  average 
routine  workload  per  aircraft  in  the  Vains  sector  and 
applying  it  to  the  traffic  in  Farmm.  Vains  and  Farmm 
are  very  similar  in  function,  operations,  and  traffic 
characteristics. 
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TABLE  3-2 


RECEP/ATF  PARAMETERS  DERIVED  FROM  SAR 


1.  Position  and  speed  history  of  all  aircraft. 

2.  Jurisdictional  control  history  of  all  aircraft. 

3.  Jurisdiction  Times  of  all  aircraft  within  each  sector. 

4.  Average  sector  Jurisdiction  Times  (sector  flight  time). 

5.  Time  under  surveillance  of  all  aircraft  within  each  sector. 

6.  Average  sector  surveillance  Times. 

7.  Number  of  aircraft  under  control  within  each  sector. 

8.  Number  of  aircraft  under  surveillance  within  each  sector. 

9.  Average  route  and  arc  transit  Times. 

10.  Number  of  aircraft  on  each  route  and  arc. 


11.  Coded  keyboard  activity  frequency  counts  by  control  position 


3.4  SUMMARY  OF  RECEP  DATA 


Following  the  conclusion  of  the  validation  tests,  RECEP 
workload  was  calculated  for  specific  time  intervals, 
eacn  of  15  minutes  duration.  Twenty-four  intervals 
were  selected  for  each  of  the  five  wor kpace-rated 
sectors  and  eight  intervals  were  selected  for  the 
remaining  six  sectors.  For  the  wor kpace-rated  sectors, 
the  selection  was  based  on  the  highest  workpace  ratings 
while  satisfying  uniform  test  conditions.  For  the 
other  sectors,  the  selection  was  based  on  the  highest 
number  of  aircraft  in  the  sector  during  the  first  two 
days  of  testing.  Two  of  the  intervals  selected  for  the 
Rockford  Sector  were  later  rejected  when  it  was 
discovered  that  all  O'Hare  arrivals  traversing  Dubuque 
and  Farmm  were  being  displayed  on  the  Rockford  Plan 
View  Display.  This  was  not  a  normal  operating  mode  and 
had  the  effect  of  greatly  inflating  the  Rockford 
surveillance  workload.  The  reason  for  it  is  unknown  so 
the  two  intervals  were  rejected. 

Table  3-3  summarizes  the  RECEP  workload  values  obtained 
for  each  of  the  time  intervals  selected.  The  table 
includes  values  for  routine,  surveillance  and  conflict 
workload  as  well  as  total  workload.  The  units  of 
workload  are  in  man-minutes  per  interval.  In  addition, 
the  table  contains  the  mean  value  for  the  number  of 
aircraft  under  control  during  each  interval,  and  the 
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TABLE  3.3  WORKLOAD-AIRCRAFT-WORKPACE  SUMMARY 
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TABLE  3.3  WORKLOAD-AIRCRAFT-WORKPACE  SUMMARY  (Cont.) 
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3.67 

2015 

3.75 

2.19 

1.23 

7.17 

7.00 

2.33 

2030 

1.95 

1.88 

0.90 

4.72 

6.00 

2.33 

2045 

3.41 

3.13 

2.50 

9.03 

10.00 

4.67 

2/28  2130 

4.49 

2.81 

2.00 

9.30 

9.00 

2.00 

2145 

5.82 

3.44 

3.00 

12.26 

11.00 

4.33 

2200 

3.34 

2.81 

2.00 

8. 15 

9.00 

2.00 

2215 

6.57 

2.81 

2.00 

11.39 

9.00 

2.67 

S/L-  2000 

1.25 

1.25 

0.40 

2.90 

4.00 

2.33 

2015 

3.36 

2.19 

1.23 

6.77 

7.00 

3.00 

2030 

2.09 

1.88 

0.90 

4.87 

6.00 

2.00 

2045 

2.62 

2  .  19 

1.23 

6.04 

7.00 

2.00 

3/1  2130 

4.52 

2.19 

1.23 

7.94 

7.00 

2.33 

2145 

4.19 

2.19 

1.23 

9.61 

7.00 

2.00 

2200 

4.47 

2.81 

2.00 

9.28 

9.00 

2.00 

2215 

4.31 

2.81 

2.00 

9.12 

9.00 

2.00 

3/3  1830 

4.47 

3.13 

2.50 

10.09 

10.00 

4.00 

1845 

3.26 

2.19 

1.23 

6.68 

7.00 

3.33 

1  900 

2.50 

1.88 

0.90 

5.28 

6.00 

3.33 

1915 

2.04 

1.56 

0.60 

4.20 

5.00 

2.67 

1/3  2130 

3.64 

2.50 

1.60 

7.74 

8.00 

3.67 

2145 

5.50 

3.44 

3.00 

11.94 

11.00 

3.67 

2200 

3.  52 

2.50 

1,60 

7.62 

8.00 

2.67 

2215 

3.26 

2.19 

1.23 

6.68 

7.00 

3.33 

^Interval  Start  Time  (GMT) 
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FABLE  3.3  WORKLOAD-AIRCRAFT-WORKPACE  SUMMARY  (Cont . ) 


2/28  18  jo 

1845 


i/J  2  l  10 
J  1  4  5 


ROUTINE. 

WORKLOAD 

(M-MIN) 


3/3 

2000 

2015 

2010 

2045 

Start 

T  ime  ( 

SURV. 
WORKLOAD 
( M-MIN) 


CONFLICT 

workload 

(M-MIN) 


5-4  1 

3.75 

7.87 

5 . 00 

11.51 

5.61 

7.  10 

4.69 

5.29 

1.75 

7.08 

5 . 00 

10.41 

5.61 

6.78 

4.38 

11.12 

5.6  1 

0,43 

4.38 

4.99 

4 . 38 

4.91 

4.38 

8.82 

3.44 

3.33 

3.13 

9.02 

5 . 00 

9.56 

5.94 

TOTAL 

WORKLOAD 

(M-MIN) 


CONTROLLKD 
AIRCRAFT 
( MF.AN ) 


6 

16 

7 

9  5 

i(J 

60 

9 

54 

7 

95 

7 

42 

8 

48 

9 

01 

LIT)' 


TABLE  3.3  WORKLOAD-AIRCRAFT-WORKPACE  SUMMARY  (Cont.) 


SECTOR 

DATE/ 

TIME* 

ROUTINE 

WORKLOAD 

(M-MIN) 

SURV. 

WORKLOAD 

(M-MIN) 

CONFLICT 

workload 

(m-min) 

TOTAL 

WORKLOAD 

(M-MIN) 

CONTROLLED 

AIRCRAFT 

(MEAN) 

WORK!’ ACE 
/sCALKlX 

V  1-7  ) 

FAR  MM 

2/2K  1830 

7.84 

4.69 

1 .20 

13.73 

8.10 

4.33 

184  5 

12.00 

7.19 

2.85 

22.04 

12.42 

6.33 

1900 

8.3b 

5.00 

1.38 

14.74 

8.64 

5.67 

191  5 

4.18 

2.50 

0.35 

7.03 

4.32 

2.33 

2/28  2130 

9.93 

5.94 

1.95 

17.82 

10.26 

5.33 

2145 

8.36 

5.00 

1.38 

14.74 

8.64 

4.33 

2200 

7.32 

4.38 

1.06 

12.76 

7.56 

3.67 

2215 

8.3b 

5.00 

1.38 

14.74 

8.64 

3.67 

.1/1  1830 

3.66 

2.19 

0.26 

6.11 

3.78 

4.00 

184  5 

f  7.84 

4.69 

13.  73 

8.10 

5.67 

1900 

8.36 

5.00 

WBM 

14.  74 

8.64 

5.67 

1915 

5.75 

3.44 

0.65 

9.84 

5.94 

3.67 

3/1  2130 

7.32 

4.32 

1.06 

12.76 

7.56 

6.00 

2145 

5.75 

3.44 

0.65 

9.84 

5.94 

3.67 

2200 

8.89 

5.31 

1.56 

15.76 

9.18 

5.33 

2215 

7.32 

4. 38 

1.06 

12.76 

7.56 

2.33 

3/2  1830 

3.66 

2.  19 

0.26 

6.11 

3.78 

2.00 

1845 

9.93 

5.94 

1.95 

17.82 

10.26 

6.00 

1900 

8.89 

5.31 

1.56 

15.76 

9.18 

5.67 

1915 

4.18 

2.50 

0.35 

7.03 

4.32 

2.67 

3/2  2130 

9.41 

5.63 

1 .  75 

16.79 

9.  72 

5.33 

2145 

8.89 

5.31 

1.  56 

15.76 

9.18 

6.00 

2  200 

7.32 

4.38 

1.06 

12.76 

7.56 

4.67 

2215 

6.27 

3.75 

0.  78 

10.80 

6.48 

4.33 

*lMtf.*rva]  Start  Time  (GMT) 
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TABLE  3.3  WORKLOAD-AIRCRAFT-WORKPACE  SUMMARY  (Cont.) 


SECTOR 

DATE/ 

T I  ME* 

ROUTINE 

WORKLOAD 

(M-MIN) 

SURV. 

WORKLOAD 

(M-MIN) 

CONFLICT 

WORKLOAD 

(M-MIN) 

T(  >TA  L 

WORKLOAD 

(M-MIN) 

CONT ROLLED 
AIRCRAf I 
( MEAN ) 

W> JRKl'Ai.  E 
/S-  Al.E.j  \ 

V  ‘“7  1 

IMA 

2/28 

18  30 

6.78 

3.13 

2.00 

11.91 

9 . 2  0 

5.3  3 

1845 

6. 40 

2 .  50 

1.30 

10.26 

7.  36 

3.3  3 

1  f*00 

5.86 

3 . 44 

2.46 

1  1  .  76 

l  0  .  1  2 

3 .  oo 

1915 

1.84 

2.19 

1.00 

5.03 

6.44 

3.00 

2/28 

2130 

10.07 

5.31 

5.87 

21.25 

15.64 

6.67 

2145 

7 . 02 

3 . 44 

2 . 46 

12.91 

10.12 

'4.67 

2200 

6.94 

3.  75 

2.9J 

13.62 

1 1.04 

3.00 

2215 

6.43 

3.  1  3 

2 . 00 

11.55 

9. 2U 

4.  33 

1/2 

181  5 

5.64 

1.56 

0.50 

7  .  70 

4.60 

1.00 

1830 

3 . 6  J 

1 .88 

0.  73 

6.24 

5.52 

2.3  3 

1845 

3.70 

2.81 

1.63 

8.  16 

8.28 

4.33 

1 900 

5 . 94 

1 . 44 

2.46 

11.84 

10.12 

4.  33 

i/2 

2130 

6.2  3 

2.81 

1.65 

10.70 

8.28 

; .  00 

2145 

5.50 

2 . 50 

1.30 

9.30 

7.36 

3.67 

2200 

7.23 

2.81 

1.65 

11.71 

8.28 

4.6  7 

2215 

4.52 

3.13 

2.00 

9.65 

9.20 

5.00 

i/3 

1830 

3.  57 

2.50 

1 .  30 

7.37 

7.36 

2 .  b  7 

1845 

7.24 

4.06 

3.40 

14.70 

11.96 

4. 33 

1 900 

6.17 

4.06 

3.40 

13.63 

11.96 

4.67 

1915 

10.  59 

3.44 

2.46 

16.49 

10.  12 

6 . 00 

3/3 

2000 

3.62 

1.56 

0.  50 

5.68 

4.60 

4 . 00 

2015 

2.42 

1.56 

0.  50 

4.48 

4.60 

2.6  7 

2030 

2.  76 

2.81 

1.65 

7.22 

8.28 

4.33 

2045 

7.18 

4.06 

3.40 

14.64 

11.96 

5.67 

*  I  Ut  t’TVfl 

1  Start 

Time  (<IMT) 

40 
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TABLE  3.3  WORKLOAD-AIRCRAFT-WORKPACE  SUMMARY  (Cont.) 


SECTOR 

DATE  / 

TIME* 

ROUTINE 

WORKLOAD 

(M-MIN) 

*UkV. 

WORKLOAD 

(M-MIN) 

CONFLICT 

WORKLOAD 

(M-MIN) 

TOTAL 

WORKLOAD 

(M-MIN) 

CONTROLLED 

AIRCRAFT 

(MEAN) 

WORKPACE 

/scaled\ 

\  I'?  ) 

JOT 

2/28  2115 

5.75 

3.44 

0.06 

9.25 

5.5 

N/A 

2130 

4.81 

3.44 

0.06 

8.31 

5.5 

2145 

4.23 

3.75 

0.08 

8.06 

6.0 

2200 

6.88 

3.13 

0.05 

10.05 

5.0 

1/1  1815 

7.77 

5.31 

0.15 

13.23 

8.5 

1830 

7.82 

6.25 

0.21 

14.28 

10.0 

1845 

7.26 

5.31 

0.15 

12.73 

8.5 

1900 

6.50 

5.00 

0.13 

11.63 

8.0 

DBQ 

2/28  1815 

6.50 

5.31 

0.95 

12.76 

10.71 

N/A 

1830 

9.26 

5.94 

1 . 20 

16.40 

11.97 

1845 

6.79 

3.75 

0.47 

11.01 

7.56 

1900 

2.41 

1.25 

0.05 

3.71 

2.52 

3/1  1815 

1.76 

2.19 

0.16 

4.11 

4.41 

1810 

4.39 

3.75 

0.47 

8.61 

7.56 

1845 

7.55 

4.69 

0.74 

12.98 

9.45 

1900 

3.96 

2.81 

0.27 

7.04 

5.67 

IOW 

2/28  2145 

2.73 

2.50 

0.23 

5.46 

6.00 

N/A 

2200 

2.63 

3.44 

0.44 

6.51 

8.25 

2215 

4.09 

4.38 

0.71 

9.18 

10.50 

2230 

3.23 

4.06 

0.61 

7.90 

9.75 

3/1  1815 

4.84 

5.94 

1.30 

12.08 

14.25 

1810 

7.21 

5.63 

1.17 

14.02 

13.50 

1845 

,3.65 

3.44 

0.44 

7.53 

8.25 

1900 

1.38 

1.56 

0.10 

3.04 

3.75 

♦Interval  Start  Time  (GMT) 

47 


'  / 

/ 


TABLE  3.3  WORKLOAD-AIRCRAFT-WORKPACE  SUMMARY  (Cont.) 


SECTOR 

DATE/ 

TIME* 

ROUTINE 

WORKLOAD 

(M-MIN) 

SURV. 

WORKLOAD 

(M-MIN) 

CONFLICT 

WORKLOAD 

(M-MIN) 

TOTAL 

WORKLOAD 

(M-MIN) 

CONTROLLED 

AIRCRAFT 

(MEAN) 

WORKPACE 
/SCALEI  \ 

l  »-*  ) 

WDPT 

2/28 

1900 

3.91 

2.81 

0.33 

7.05 

5.04 

N/A 

1915 

6.84 

5.00 

1.04 

12.88 

8.96 

1930 

8.49 

5.31 

1.18 

14.98 

9.52 

1945 

7.79 

5.63 

1.32 

14.  73 

10.08 

2/28 

2130 

4.09 

3.44 

0.49 

8.02 

6.16 

2145 

3.76 

2.50 

0.26 

6.52 

4.48 

2200 

5.25 

4.38 

0.80 

10.43 

7.84 

2215 

6.43 

5.31 

1.18 

12.92 

9.52 

RF[) 

2/28 

1815 

2.31 

2.50 

1.04 

5.85 

6.96 

N/A 

■k'k 

1830 

2.64 

6.56 

2.56 

11.77 

18.27 

■kit 

1845 

2.83 

7.19 

3.58 

13.59 

20.01 

1900 

4.75 

3.13 

1.72 

9.60 

8.70 

2/28 

2130 

5.40 

3.75 

3.05 

12.20 

10.44 

2145 

4.92 

3.75 

3.05 

11.72 

10.44 

2200 

2.31 

1.88 

0.76 

4.95 

5.22 

2215 

4.  12 

2.81 

1.72 

8.65 

7.83 

MCK 

2/28 

2115 

3.11 

2.81 

1.14 

7.07 

7.38 

N/A 

2130 

2.95 

3.75 

2.03 

8.73 

9.84 

2145 

5.06 

4.06 

2.39 

11.51 

10.66 

2200 

3.18 

2.50 

0.90 

6.  58 

6.  56 

3/1 

1815 

3.43 

3.13 

mem 

7.96 

8.20 

1830 

5.39 

5.00 

WBm 

13.99 

13.12 

1845 

1.71 

3.13 

1 .40 

6.24 

8.20 

1900 

1.40 

1.56 

- 

0.35 

3.31 

4.10 

^Interval  Start 

Time  (GMT) 

** Interval  rejected  due  to  aberrant  conditions 

workpace  rating,  where  applicable.  Workpace  ratings 
are  on  a  numerical  scale  from  1  to  7  where  7  represents 
"very  heavy"  workpace.  The  average  value  of  workload 
over  the  166  test  intervals  is  11.1  man-minutes. 
Routine,  surveillance  and  conflict  workload  account  for 
51%,  35%,  and  14%  of  the  total,  respectively. 

Routine  workload  was  analyzed  by  dividing  it  into  the 
five  components:  air/ground  controller  speaking, 
air/ground  controller  listening,  interphone,  FDP 
operations,  and  flight  strip  activities.  Workload 
values  for  each  component  were  obtained  for  all  test 
intervals.  From  these  data  a  maximum,  minimum,  and 
mean  value  were  obtained  for  each  component.  A  uniform 
pattern  emerged  for  all  sectors.  Figure  3-1 
illustrates  the  results  for  the  Vains  Sector,  which  are 
typical  of  all  sectors.  Air/ground  controller  speaking 
was  always  the  highest  value,  accounting  for  over  30% 
of  the  total  routine  workload. 

SAR  data  for  all  four  days  of  testing  were  analyzed  to 
determine  the  average  length  of  time  aircraft  were 
under  the  surveillance  of  each  sector  and  also  under 
the  jurisdiction,  or  control,  of  each  sector.  The 
average  jurisdiction  interval  is  equivalent  to  sector 
flight  time.  Sector  flight  time  is  of  interest  when 
converting  RECEP  flow  rate  to  hourly  flow  rate  and  also 
in  selecting  the  duration  of  RECEP  test  intervals  and, 

49 


'  / 


S3Ii.IAIJ.3V 

dials 

1H9IU 


SNOiivaido 

daa/dQj 


wwooaaiNi 


9NIN31SI3 

b3110aiN0D 

9/V 


9NI»V3dS 

b3310aiN03 

9/V 


CNIW  SI  a3d  -NIW-NVW)  avoixaoM 


FIGURE  3-1.  COMPONENTS  OF  ROUTINE  WORKLOAD 


in  the  case  of  conflict  prevention,  in  selecting 
traffic  samples.  Table  3-4  lists  both  the  surveillance 
and  jurisdiction  intervals  for  all  sectors  for  each  of 
the  four  days  of  testing  as  well  as  an  average  value 
for  all  days.  The  table  lists  in  parentheses  the 
number  of  aircraft  on  which  each  value  was  based.  The 
surveillance  interval  was  obtained  by  measuring  the 
duration  of  time  that  all  aircraft  symbols  with  full 
data  blocks  were  displayed  on  the  Plan  View  Display 
regardless  of  jurisdictional  control.  Thus,  pointout 
aircraft  symbols  are  included.  The  jurisdiction 
interval  was  obtained  by  measuring  the  duration  of  time 
between  FDP  hand-in  and  hand-out  events.  This 
necessarily  involves  a  smaller  sample  of  aircraft  since 
some  handoffs  do  not  involve  computer  flight  data 
processing.  This  is  particularly  true  for  low  altitude 
sectors.  As  can  be  seen  from  the  table,  the  sector 
flight  times  vary  from  7.0  to  20.3  minutes  with  an 
average  value  of  13.35  minutes. 

3.5  SUMMARY  OF  ATF  DATA 

An  empirical  traffic  sample  from  noon  to  5  PM  on  the 
first  day  of  testing  was  chosen  for  the  construction  of 
an  ATF  model  of  the  eleven  sector  area.  Over  700 
aircraft  traversed  the  area  during  this  five-hour 
period.  The  time  and  flight  path  histories  of  all 
aircraft  were  extracted  from  SAR  data  and  a  route 
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structure  was  determined  from  the  flight  paths.  A 
total  of  83  routes  were  defined.  Each  route  was 
divided  into  arcs  which  correspond  to  the  segment  of 
the  route  traversing  individual  sectors.  Arc  transit 
times  were  computed  based  on  the  average  time  duration 
of  all  aircraft  on  each  arc.  Arc  transit  times  varied 
from  3  to  28  minutes.  Aircraft  were  assigned  to  the 
appropriate  route.  The  entry  times  of  aircraft 
entering  routes  were  extracted  from  SAR  data  in  groups 
of  15-minute  intervals.  The  ATF  computer  program 
randomly  distributes  the  actual  entry  time  within  each 
interval . 

The  coefficients  for  the  ATF  workload  equation  were 
determined  by  performing  a  least  squares  regression 
analysis  on  the  values  obtained  for  total  RECEP 
workload  versus  number  of  aircraft  under  control  from 
each  of  the  RECEP  test  intervals.  A  second  order 
function  with  a  zero  constant  term  was  assumed  in 
performing  the  regression.  Table  3-5  lists  the 
coefficients  obtained  for  each  of  the  eleven  sectors  as 
well  as  the  generalized  ATF  workload  equation  with  the 
units  of  each  term.  RECEP  workload  values  are  measured 
in  man-minutes  per  15  minutes  whereas  the  ATF  model 
computes  workload  in  man-seconds  per  minute. 
Therefore,  a  factor  of  60/15  is  applied  in  computing 
the  coefficients. 
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TABLE  3-5.  ATF  WORKLOAD  EQUATION  COEFFICIENTS 


SECTOR 

K1 

K2 

JOT 

5.73 

0 

BDF 

4.60 

0.10 

DBQ 

4.52 

0.068 

IOW 

3.08 

0.044 

ML  I 

2.73 

0.15 

WDPT 

4.96 

0.088 

VAINS 

4.74 

0.25 

RFD 

2.  58 

0.195 

FARMM 

6.2 

0.074 

PIA 

4.17 

0.073 

MCK 

2.82 

0.1 

WL  =  KX  N  +  K2  N2 

WHERE:  WL  =  WORKLOAD  IN  MAN 

-SECONDS  PER  MINUTE 

Kx  =  COEFFICIENT  IN  MAN-SECONDS  PER 

MINUTE  PER  AIRCRAFT 

K2  =  COEFFICIENT  IN  MAN-SECONDS  PER 

MINUTE  PER  (AIRCRAFT)2 

N  =  NUMBER  OF  AIRCRAFT 
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Figure  3-2  illustrates  one  usage  of  the  ATF  model  in 
evaluating  long-term  workload.  The  five-hour  traffic 
sample  was  run  into  the  program.  The  first  30  minutes 
were  considered  to  be  an  initialization  period  as 
aircraft  entered  routes  and  flowed  from  one  sector  to 
another,  leaving  270  minutes  for  meaningful  measures. 
Workload  was  printed  out  minute-by-minute  and  the 
percent  of  time  that  workload  was  above  a  given  level 


was  determined  for  each 

sector . 

The  shape  of 

the 

function 

shown  in  the 

figure 

is  typical  for 

all 

sectors. 

A  comparison 

of  sectors  provides 

an 

indication  of  relative 

sector 

workload  over 

long 

per iods 

of  time.  The 

figure 

also  indicates 

the 

workload  level  which  corresponds  to  an  average  workpace 
rating.  This  was  determined  from  the  controller  survey 
data  which  equated  the  number  of  aircraft  in  the  sector 
with  various  workpace  ratings.  The  number  of  aircraft 
was  then  converted  to  a  RECEP  workload  value  by  means 
of  the  RECEP  function  for  workload  versus  aircraft. 

3.6  EXAMPLE  OF  THE  DETERMINATION  OF  POTENTIAL  CONFLICT 
WORKLOAD  COEFFICIENTS 

The  Bradford  (BDF)  high  altitude  sector  has  been 
selected  to  demonstrate  the  methodology  used  in 
estimating  potential  conflict  workload  coefficients. 
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FIGURE  3-2.  WORKLOAD  VERSUS  PERCENT  OF  TIME 


A  major  part  of  BDF  traffic  consists  of  southwest 
arrivals  to  O'Hare  (ORD) .  There  is  also  a  southwest 
departure  route  from  ORD.  The  remaining  traffic  is 
essentially  overflights.  Figure  3-3  shows  the  flight 
plan  ground  tracks  for  one  hour  of  aircraft  entering 
BDF.  Figure  3-4  shows  the  actual  SAR  ground  tracks  for 
the  same  aircraft. 

Table  3-6  lists  all  the  flights  entering  BDF  along  with 
ground  speed  and  altitude  information.  Figures  3-5 
through  3-8  are  plots  of  the  SAR  ground  tracks  for  the 
four  15-minute  periods.  The  conflict  and  overtake 
equations  of  Section  2.1.3  were  used  to  calculate  the 
expected  number  of  conflicts  using  Table  3-6  and  the 
appropriate  figure,  the  expected  number  of  conflicts 
can  now  be  calculated. 

1800  -  1815 


Expected  No. 

of 

Overtakes 

2  and  3 

0 

=  0.154 

Expected  No. 

of 

Crossings 

4  with  2  and  . 

3  C  =  0.261 

1815  -  1830 

Expected  No. 

of 

Over  takes 

6,  7,  and 

8 

0  *  0.050 

Expected  No. 

of 

Crossings 

9  with  10 

and 

12  C  *  0.495 

1830  -  1845 
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FIGURE  3-3.  TRAFFIC  HANDED  OFF  TO  BDF  1800-1900  GMT,  2-28-78 
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TABLE  3-6.  AIRCRAFT  ENTERING  BDF  1800-1900  GMT,  February  28,  1978 
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FIGURE  3-8.  GROUND  TRACK  OF  AIRCRAFT  ENTERING  BDF  1845- 


Expected  No.  of  Overtakes 
None?  14  and  15  have 
the  same  speed 
Expected  No.  of  Crossings 


13 

with 

14 

and 

15 

C  -  .315 

13 

with 

16 

C  =  .165 

16 

with 

14 

and 

15 

C  -  .195 

Total 

C  -  .675 

1845  -  1900 

Expected  No.  of  Overtakes 


21 

,  22, 

23 

and 

24  0  - 

0.527 

25 

and 

26 

0  » 

0.099 

Total  0  * 

0.626 

Expected 

NO« 

of 

Crossings 

21 

,  22, 

23 

and 

24 

with 

25 

and 

26  C 

*  1.396 

21 

,  22, 

23 

and 

24 

with 

20 

C 

*  0.644 

25 

and 

26 

with 

20  C 

»  0.354 

19  with  27  C  -  0.253 
Total  C  *  2.647 

As  car.  be  seen  from  the  above,  the  expected  number  of 
crosing  conflicts  can  exceed  1.0.  In  some  cases,  the 
expected  number  of  conflicts  is  calculated  to  be 
greater  than  the  minimum  flow  rate  of  the  two 
intersecting  routes.  Obviously,  this  is  illogical 
since  the  aircraft  on  at  least  one  of  the  flight  paths 


coefficients  were  obtained  by  deriving  a  single 
workload  coefficient.  The  reason  for  this  is  that 
there  are  time  periods  when  the  expected  number  of 
crossing  and/or  overtake  conflicts  is  zero  and  the  data 
sample  becomes  significantly  smaller. 

By  performing  a  second  order  least  squares  regression 
on  the  20  samples  of  conflict  workload  for  BDF,  a  mean 
workload  coefficient  of  0.025  man-minutes  per  15 
minutes  per  aircraft  was  obtained.  Table  3-7  contains 
the  coefficients  for  all  eleven  sectors. 


67 


-  / 


TABLE  3-7 


CONFLICT  WORKLOAD  COEFFICIENTS 


Sector 

Conflict  Workload 

Coef f icient 

JOT 

0.0021 

BDF 

0.025 

DBQ 

0.0082 

IOW 

0.0065 

MLI 

0.02  5 

WDPT 

0.013 

VAINS 

0.018 

RFD 

0.028 

FARMM 

0.018 

PIA 

0.024 

MCK 

0.021 

NOTE:  Conflict  Workload  Coefficient  units  are  man-minutes 

per  fifteen  minutes  per  (aircraft)^. 

4. 


VALIDATION  ANALYSIS  OF  WORKLOAD  DATA 


4.1  INTRODUCTION 

The  principal  purpose  of  this  validation  analysis  is  to 
determine  whether  a  measure  of  controller  workload, 
derived  offline  from  automatic  recordings  according  to 
specifications  for  the  RECEP/ATF  models,  agrees  with 
workload  estimates  made  by  controllers  themselves. 

A  fundamental  characteristic  of  the  concept, 
"workload,"  is  that  it  concerns  the  magnitude  of  an 
internal  condition  experienced  by  the  working 
individual.  Workload  is  not  an  observable  effect  about 
which  all  observers  are  certain  to  agree.  Observers' 
workpace  ratings  are,  nonetheless,  being  used  in  this 
analysis  as  a  standard  for  evaluating  the  validity  of  a 
workload  measure  that  is  computed  from  offline 
recordings.  Although  we  might  simply  assume  our  sample 
of  workpace  ratings  is  valid,  and  then  apply  it  as  an 
evaluation  criterion,  our  conclusions  would  be  more 
acceptable  if  we  assess  the  validity  of  the  ratings 
themselves  and  demonstrate  objectively  that  they  are 
probably  valid. 

There  is  no  certain  measure  of  the  true  value  of  the 
workload  that  controllers  actually  experience  while 
they  are  observed,  but  we  can  at  least  test  the 
consistency  between  the  ratings  and  other  information 
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acquired  via  questionnaires  regarding  typical  sector 
differences  in  workload  and  control  difficulty.  The 
demonstrated  logical  consistency  between  controllers' 
ratings  and  controllers'  questionnaire  responses  can 
then  provide  assurance  that  the  subjective  validation 
criteria  are  probably  valid  for  evaluating  the  workload 
measure  derived  offline  some  time  after  the  work 
occurred . 

Our  general  approach  is  to  perform  various  analyses 
that  test  the  logical  consistency  among  workload 
measures  derived  from  three  different  sources:  "Rated 
Workload"  derived  from  workpace  ratings,  "Judged 
Workload"  derived  from  questionnaire  responses,  and 
"Computed  Workload"  derived  offline  from  data 
recordings.  Whereas  workpace  ratings  were  made  by 
controllers  who  observed  the  work  as  it  occurred, 
questionnaire  responses  were  made  by  controllers  who 
referred  to  their  past  experience  in  the  studied 
sectors  in  order  to  judge  typical  levels  of  workload 
and  control  difficulty.  Although  Rated  Workload  and 
Computed  Workload  both  pertain  operationally  to  work  as 
it  occurs,  relationships  involving  these  measures  can 
presumably  be  used  to  estimate  typical  workload 
conditions  in  selected  sectors.  Measures  of  Judged 
workload  permit  us  to  generalize  explicitly  beyond  the 
sample  of  data  that  we  collect.  The  three  kinds  of 
measures  are  thus  applied  in  a  series  of  subanalyses 
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whose  results  and  implications  converge  logically  to 


support  two  conclusions:  (1)  Computed  Workload  can 

provide  a  valid  index  of  controller  workload  as  it 
occurs,  and  (2)  Computed  Workload  can  provide  a  valid 
estimate  of  typical  workload  conditions  in  individual 
enroute  sectors. 

4.2  SUMMARY  STATISTICS  FOR  SAMPLED  SECTOR  ACTIVITIES 

Statistics  that  summarize  control  activities  that 
occurred  during  15  minute  intervals  for  which  Computed 
Workload  was  derived  are  given  in  Table  4-1  for 
workpace  rated  sectors  and  in  Table  4-2  for  unrated 
sectors.  The  statistics  are  for  the  following 
variables : 

1.  Traffic  Flow  Rate,  the  number  of  aircraft  that 
entered  the  sector's  jurisdiction. 

2.  Aircraft  Under  Control,  the  average  number  of 
aircraft  for  which  full  data  blocks  were  displayed 
either  because  the  aircraft  was  under  the  sector's 
jurisdiction,  because  the  aircraft  had  been  pointed 
out  by  another  sector,  or  because  the  aircraft  had 
been  selected  for  monitoring  by  the  controllers  in 
the  sector. 
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TABLE  4-1.  STATISTICS  FOR  VARIABLES  IN  RATED  SECTORS 
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TABLE  4-2.  STATISTICS  FOR  VARIABLES  IN  UNRATED  SECTORS 


3.  Total  Workload,  an  index  of  the  amount  of  mental 
activity  performed  by  the  R  controller.  Total  work 
is  estimated  by  Rated  and  Computed  Workload  in 
rated  sectors  ,  but  only  by  Computed  Workload  in 
unrated  sectors.  Rated  Workload  is  an  average 
value  ranging  from  1  to  7,  integers  that  had  been 
assigned  to  controllers'  ratings  in  order  to 
represent  ratings  from  "Very  Light"  to  "Very 
Heavy."  Computed  Workload  is  the  sum  of  minutes  for 
Routine,  Surveillance,  and  Conflict  Prevention 
workload  components. 

In  the  data  for  all  11  of  the  studied  sectors,  the  mean 
Aircraft  Under  Control  ranged  between  7.1  and  9.3 
aircraft.  Also,  the  mean  workpace  in  the  rated  sectors 
ranged  from  2.8,  or  approximately  "Below  Average,"  in 
MLI ,  to  5.1,  or  approximately  "Above  Average,"  in 
VAINS .  Although  the  average  values  for  different 
sectors  were  similar  in  magnitude,  various  systematic 
relationships  pertinent  to  this  validation  analysis  are 
present  in  the  data. 

Mean  Computed  Workload  as  a  Function  of  Mean  Workpace 

A  positive  relationship  between  Computed  Workload  and 
Rated  Workload  can  be  found  in  Table  4-1.  Figure  4-1 
indicates  that  the  least  squares  regression  line 
describing  the  empirical  relationship  has  a  positive 
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FIGURE  4-1.  MEAN  COMPUTED  WORKLOAD  AS  A  FUNCTION  OF  MEAN  WORKPACE 


slope  as  would  be  expected  if  the  two  variables  measure 
the  same  thing.  A  statistical  test  of  the  hypothesis 
that  the  product  moment  correlation  (r=0.73)  between 
the  means  is  equal  to  zero,  was  performed  using  the 
statistic , 


The  test  indicates  the  observed  relationship  among  the 
five  sectors  (N=5)  can  occur  by  chance  almost  20 
percent  of  the  time.  While  this  level  of  risk  in 
drawing  a  faulty  inference  (i.e.,  "a  positive 
relationship  exists")  may  be  acceptable  for  practical 
purposes,  the  observed  relationship  does  not 

demonstrate  that  the  two  variables  are  probably 
equivalent  in  meaning.  Other  kinds  of  analyses  are 
required  for  building  a  plausible  case  that  Rated 
Workload  and  Computed  Workload  are  equivalent  for 
estimating  controller  workload. 

The  following  analyses  show: 

1.  Mean  workpace  ratings  are  consistent  with  what  the 
raters  evidently  believed  were  typical  traffic 
levels.  This  finding  gives  us  confidence  in  the 
validity  of  the  ratings. 
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2.  Both  the  Rated  and  the  Computed  measures  of 

workload  are  more  strongly  correlated  with  Aircraft 
Under  Control  than  with  Traffic  Flow'  Rate.  If  the 
two  workload  measures  were  correlated  with 

different  measures  of  traffic  (a  conceivable 

outcome,  given  the  existing  literature) ,  we  would 
have  less  confidence  that  the  measures  reflect  the 
same  aspects  of  the  control  process. 

3.  Workload  estimates  based  on  the  Rated  and  Computed 
measures  of  workload  agree  in  showing  how  the 
studied  sectors  differ  from  each  other  in  terms  of 
control  difficulty.  This  is  evidence  that  the 
measures  are  valid  for  comparing  workload  in 
different  sectors. 

4.  When  values  of  Rated  and  Computed  workload,  for  the 
same  time  intervals  in  an  individual  sector,  are 
plotted  together  as  functions  of  clock  time  (i.e., 
successive  samples  are  given  in  the  order  of  their 
occurrence)  ,  the  values  agree  in  showing  whether 
workload  increases  or  decreases.  This  is  evidence 
that  the  measures  can  be  used  to  assess  workload 
changes  over  time  within  individual  sectors. 


After  these  validation  analyses  are  presented,  we  focus 
on  the  problem  of  estimating  the  maximum  workload  in 
individual  sectors. 
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4.3  WORKPACE  RATINGS  IN  RELATION  TO  EXPECTED  TRAFFIC  LEVELS 


It  was  noted  in  Table  4-1  that  the  mean  workpace 
ratings  ranged  from  somewhat  "Below  Average"  in  MLI  to 
"Above  Average"  in  VAINS.  Though  the  five  mean 
workpace  values  cover  a  limited  range,  they  appear  to 
be  ordered  quite  systematically  as  a  function  of  other 
var iables . 

In  this  case,  let  us  assume  that  the  mean  ratings  are 
consistent  with  typical  operations  in  the  rated 
sectors.  A  rating  of  "Average"  (assigned  the  value,  4) 
is  thus  presumably  anchored  subjectively  to  what  the 
raters  believed  was  average  in  the  long  run.  Since 
workload  is  strongly  affected  by  the  traffic  level,  as 
the  next  analysis  indicates,  a  traffic  level  less  than 
the  average  level  would  presumably  be  accompanied  by  a 
"Below  Average"  workpace  rating,  which  we  found  in  the 
data  for  MLI.  Similarly,  when  traffic  exceeds  the 
average,  workpace  would  be  "Above  Average,"  as  we  found 
in  the  data  for  VAINS.  Following  this  line  of 
thinking,  we  analyzed  the  data  to  see  whether  the 
traffic  actually  observed  was  more  or  less  than  the 
level  the  controllers  believed  to  be  average;  the 
deviation  was  then  used  to  predict  the  observers'  mean 
workpace  ratings. 
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In  the  analysis  of  data,  the  traffic  level  that  Chicago 
controllers  judged  as  producing  "Average"  workload  was 
derived  from  the  controllers'  questionnaire  responses 
(the  Appendix,  Interview  3).  Each  controller's 
workload-traffic  judgments  for  a  sector  were  described 
by  a  least  squares  regression  line  that  was  then  used 
to  estimate  the  number  of  aircraft  typically  under 
control  for  an  "Average"  workpace.  Individual 
estimates  from  five  controllers  (per  control  area)  were 
then  averaged,  and  the  mean  was  subtracted  from  the 
mean  number  of  aircraft  that  were  actually  under 
control  in  the  sector.  The  aircraft  difference,  the 
predictor  variable,  is  plotted  in  Fig.  4-2  on  the 
abscissa;  the  mean  workpace  rating  is  the  predicted 
variable. 

Figure  4-2  shows  that  mean  workpace  is  systematically 
related  to  the  difference  between  the  observed  traffic 
and  the  judged  traffic  for  an  "Average"  workload.  The 
least  squares  regression  line  describing  the  empirical 
relationship  refers  to  the  filled  points  that  represent 
all  the  workpace  and  traffic  data  collected  for  the 
study  (48  15-minute  samples);  unfilled  points 
represent  the  subset  of  data  (24  samples)  reported  in 
Table  4-1.  Figure  4-2  shows  that  similar  results  are 
obtained  whether  all  the  workpace  data  are  used  or  only 
the  subset.  From  Fig.  4-2,  we  infer  that  the  mean 
workpace  ratings  for  the  five  rated  sectors  are 
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consistent  with  what  the  raters  believed  are  typical 
traffic  levels  in  the  rated  sectors.  This,  of  course, 
is  precisely  what  we  had  hoped  would  be  true,  but  could 
not  guarantee  operationally;  much  depended  on  the 
raters  themselves. 

4.4  CORRELATIONS  BETWEEN  WORKLOAD  AND  TRAFFIC  VARIABLES 

Having  seen  that  the  mean  workpace  ratings  are  an 
orderly  function  of  the  raters'  expectations  regarding 
the  number  of  aircraft  under  control,  we  should  ask 
whether  workpace  ratings,  in  particular,  and  measures 
of  workload,  in  general,  are  strongly  related  to  one  or 
more  measures  of  traffic.  Our  more  general  concern 
with  the  relationship  between  workload  and  traffic 
stems  from  a  practical  requirement;  namely,  the  need 
to  estimate  workload  for  specified  levels  of  traffic  in 
individual  sectors. 

While  analyzing  data  from  the  Chicago  Center,  we 
discovered  a  much  stronger  correlation  between  workpace 
and  Aircraft  Under  Control  (AUC)  than  between  workpace 
and  Traffic  Flow  Rate  (TFR) .  Pursuing  this  interesting 
difference,  we  then  found  that  measures  of  workload 
derived  offline  from  automatic  recordings  are  also  more 
strongly  correlated  with  AUC  than  with  TFR  in  all  11  of 
the  studied  sectors.  This  consistency  between  Rated 
Workload  and  Computed  Workload  is  one  kind  of  evidence 
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suggesting  that  the  two  kinds  of  measures  do  indeed 
refer  to  the  same  aspects  of  the  control  process. 
After  establishing  that  this  effect  is  general  in  our 
data,  we  selected  AUC  as  the  traffic  variable  for 
precisely  predicting  workload  by  means  of  least  squares 
regression  equations  fit  to  the  data.  In  Table  4-3, 
product  moment  coefficients  of  correlation  relate  Rated 
Workload  (workpace)  and  Computed  Workload  (Total  and 
Routine)  to  TFR  and  AUC. 

4.4.1  Rated  Workload  as  a  Function  of  Traffic 

Rated  Workload  correlated  0.37-0.52  with  TFR  and 


0.49-0.89 

with 

AUC. 

In  other  words, 

1 inear 

regression 

with 

TFR 

accounted  for  up 

to  26 

percent  of  the  workpace  variance;  however, 
regression  with  AUC  accounted  for  substantially 
more,  up  to  79  percent.  In  searching  the 
literature  for  an  analogous  TFR-AUC  difference 
to  confirm  the  generality  of  our  finding,  we 
found  in  a  recent  analysis  of  data  from  47 
sectors  at  the  Boston  and  New  York  Centers  (Ref. 
8)  a  difference  of  the  same  magnitude.  The 
other  correlations  were  between  workpace  ratings 
on  a  16  category  scale  and  either  the  traffic 
count  per  hour  (r=0.49),  which  is  a  measure  of 
traffic  flow  rate,  or  the  peak  traffic  count  per 
10  minute  interval  (r«0.70),  which  is  similar  to 
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oar  measure  of  aircraft  under  control. 


The 


cited  correlations  were  for  a  group  of  different 
sectors,  whereas  ours  are  for  individual 
sectors.  The  high  degree  of  agreement,  despite 
various  procedural  differences  between  the 
present  study  and  the  recent  cited  study  , 
attests  to  the  generality  of  the  finding  that 


wor  kpace 

is  more  highly 

correlated  with  AUC 

than 

with  TFR 

• 

Computed 

Workload  as  a 

Function  of  Traffic 

Consistent  with  the 

pattern  shown 

by 

Rated 

Workload 

,  Computed 

Workload  is 

also 

more 

strongly 

related  to  AUC 

than  to  TFR. 

Table 

4-3 

shows 

that  for  Routine  Workload 

in 

rated 

sectors , 

linear  regression  accounts 

for 

less 

than  20  percent  of  the  variance  with  TFR,  but 
more  than  46  percent  with  AUC.  Again, 
substantially  more  variance  is  accounted  for 
using  AUC,  which  therefore  permits  more  precise 
workload  predictions. 

Computed  Routine  Workload,  as  well  as 
Surveillance  Workload  and  Conflict  Prevention 
Workload,  are  components  of  Computed  Total 


Workload , 

which 

also 

shows  a  stronger 

relationship 

with 

AUC. 

But  the  very  high 
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correlations  in  the  case  of  Total  Workload  are 


due,  in  part,  to  the  way  that  the  other  workload 
components  were  derived  using  AUC  (see  the 
preceding  chapter  for  details)  .  The  same 
qualification  applies  to  the  Routine  Compute^ 
Workload  in  FARMM;  this  workload  component  was 
derived  as  a  function  of  AUC  from  VAINS,  because 
voice  communications  data  for  FARMM  were 
unavailable. 


Before  this  analysis,  it  was  conceivable  that  we 
would  find  that  Computed  Workload  is  highly 
correlated  with  Traffic  Flow  Rate  and  Rated 
Workload  is  highly  correlated  with  Aircraft 
Under  Control;  these  correlations  were 
evidently  not  compared  in  previous  research. 
The  fact  that  both  measures  of  workload  are 
highly  correlated  with  the  same  measure  of 
traffic  is  important,  because  the  agreement  is 
one  kind  of  evidence  that  suggests  the  two 
workload  measures  reflect  the  same  aspects  of 
the  control  process. 


Having  established  that  the  relationship  with 
Aircraft  Under  Control  is  general,  we  selected 
this  traffic  measure  as  the  basis  for  predicting 
workload  using  least  squares  regression 
equations  fit  to  the  data.  In  the  following 
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section,  the  linear  equations  fit  to  Rated, 
Computed,  and  Judged  Workload  are  described. 

4.4.3  Linear  Workload- Traffic  Functions 

Linear  functions  relating  workpace  to  AUC  are 
given  in  Table  4-4.  The  strong  linear  relation 
indicated  by  the  high  correlations  is 
reminiscent  of  results  obtained  previously  in 
terms  of  peak  aircraft  under  control  at  a  number 
of  air  traffic  control  facilities  (e.g.,  Ref. 
9)  . 

Scatterplots  for  linear  functions  that  relate 
Computed  Total  Workload  to  AUC  are  illustrated 
in  Figs.  4-3  and  4-4,  for  rated  and  unrated 
sectors,  respectively.  Because  the  workload  and 
traffic  variables  are  very  highly  correlated  (r 
is  at  least  0.90),  :he  workload  associated  with 
a  given  level  of  traffic  can  be  estimated 
precisely. 

Linear  functions  involving  Judged  Workload, 
derived  from  questionnaire  responses  expressed 
on  the  same  scale  as  the  workpace  scale,  are 
given  in  Table  4-5.  For  each  studied  sector, 
the  workload-traffic  judgments  (the  Appendix, 
Interview  S)  of  five  controllers  were  combine  i 
into  a  single  set  of  wor kpace- tr a f f 1 c  pairs  t 
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TABLE  4-4.  WORKLOAD  AS  A  FUNCTION  OF  AIRCRAFT  UNDER  CONTROL 
FUNCTIONS  DERIVED  FROM  OBSERVERS*  WORKPACE  RATINGS 


AREA 

SECTOR 

NUMBER  OF 
OBSERVATIONS 

WORKPACE- TRAFFIC 
CORRELATION 

SLOPE 

INTERCEPT 

WEST 

WING 

ML  I 

48 

0.67 

0.25 

0.66 

24 

0.49 

0.21 

1.17 

PIA 

48 

0.77 

0.36 

0.71 

24 

0.77 

0.31 

1.65 

WEST 

TERMINAL 

VAINS 

48 

0.85 

0.62 

-0.18 

24 

0.89 

0.68 

-0.37 

FARMM 

48 

0.85 

0.52 

0.54 

24 

0.78 

0.46 

0.93 

WEST 

HIGH 

BDF 

48 

0.75 

0.37 

-0.27 

24 

0.67 

0.35 

0.16 

Note: 

Two  controller  raters  alternated  between  the  two  West  Wing  sectors. 

Two  controllers  alternated  between  the  two  West  Terminal  sectors. 

One  controller  rated  the  West  High  sector. 

Sectors  are  listed  in  each  area  by  increasing  judged  control  difficulty. 
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TOTAL 

WORK  (MINUTES) 


AIRCRAFT 


SLOPE  1.59 
INTERCEPT  -1.75 
CORRELATION  0.90 


SLOPE  1.30 
INTERCEPT  -2.41 
CORRELATION  0.96 


SLOPE  2.19 
INTERCEPT  -3. 82 
CORRELATION  0.90 


SLOPE  1.82 
INTERCEPT  -0.93 
CORRELATION  1.00 


SLOPE  1.36 
INTERCEPT  -1.21 
CORRELATION  0.92 


FIGURE  4-3.  COMPUTED  TOTAL  WORKLOAD  AS  A  FUNCTION  OF 

AIRCRAFT  UNDER  CONTROL  IN  RATED  SECTORS 
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which  a  linear  least  squares  regression  function 
was  fit.  Table  4-5  gives  the  slopes, 
intercepts,  and  correlations;  correlations 
ranging  from  0.79  to  0.92  indicate  very  precise 
linear  relationships. 

These  three  groups  of  workload-traffic  functions 
will  be  used  to  estimate  workload  values  in 
relation  to  additional  judgments  that 
controllers  made  regarding  sector  control 
difficulty. 

4.5  WORKLOAD  AS  A  FUNCTION  OF  SECTOR  CONTROL  DIFFICULTY 

The  concept  of  workload  is  associated  with  the  idea  of 
work  difficulty,  at  least  to  the  extent  that  a  more 
difficult  task  can  cause  an  individual  to  work  harder. 
Working  harder,  the  individual  might  feel  more  heavily 
"loaded*  physically  or  mentally.  During  informal 
conversations  with  air  traffic  controllers  regarding 
their  control  areas,  the  controllers  sometimes 
differentiate  among  sectors  in  terms  of  the  sectors’ 
relative  control  difficulty  as  a  cause  for  differences 
in  workload.  In  order  to  obtain  formal  data  that  might 
be  used  as  a  tool  for  analyzing  the  validity  of  the 
workload  measures  collected  in  this  study,  we  asked 
controllers  to  judge  the  relative  difficulty  of  the 
studied  sectors  in  their  control  areas  (the  Appendix, 
Interview  1) . 
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TABLE  4-5.  WORKLOAD  AS  A  FUNCTION  OF  AIRCRAFT  UNDER  CONTROL 
FUNCTIONS  DERIVED  FROM  QUESTIONNAIRE  JUDGMENTS 


AREA 

SECTOR 

WORKLOAD- TRAFFIC 
CORRELATION 

SLOPE 

INTERCEPT 

WEST 

WING 

ML  I 

0.84 

0.29 

1.27 

PIA 

0.91 

0.36 

0.47 

WEST 

TERMINAL 

RFD 

0.85 

0.42 

0.61 

WDPT 

0.79 

0.37 

1.57 

VAINS 

0.86 

0.31 

2.10 

FARMM 

0.92 

0.41 

1.27 

WEST 

HIGH 

MCK 

0.86 

0.29 

0.23 

IOW 

0.84 

0.30 

0.68 

JOT 

0.86 

0.41 

0.53 

BDF 

0.85 

0.36 

0.66 

D3Q 

0.84 

0.34 

0.42 

Note: 

In  each  area,  five  journeyman  controllers  judged  the  typical 
workload  (workpace  scale)  produced  by  specified  numbers  of 
aircraft  displayed  simultaneously  on  the  PVD  in  each  named 
sector.  The  above  parameters  are  for  linear  least  square 
functions  fit  to  the  combined  ratings.  Sectors  are  listed 
in  each  area  by  increasing  judged  control  difficulty. 
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4.5.1  Measures  of  Relative  Sector  Difficulty 

The  controllers  assigned  each  studied  sector  to 
a  position  on  a  seven  category  scale  (analogous 
to  the  workpace  scale)  according  to  the  sector's 
relative  control  difficulty.  Each  controller 
was  also  asl.ed  to  explain  in  his  own  words  why 
he  believes  the  sectors  differ.  Later,  we 
assigned  integers  of  1-7  to  the  controllers' 
position  responses.  The  responses  for  each 
sector  were  averaged  over  the  five  controllers 
who  were  interviewed. 

Mean  judgments  for  each  sector  are  plotted  in 
Fig.  4-5;  the  dispersion  around  the  mean  is 
based  on  the  standard  deviation  of  the  five 
controllers'  judgments.  Why  the  sectors  differ 
as  shown  is  suggested  in  Table  4-6  by  the  list 
of  the  difficulty  factors  that  the  controllers 
cited.  The  number  of  controllers  (maximum  of 
five  in  each  area)  who  cited  each  factor  is 
given.  We  see  in  the  table  that  the  total 
number  of  difficulty  citations  for  a  sector 
tended  to  be  higher,  the  higher  the  mean  rating 
of  sector  difficulty.  According  to  the 
controllers'  difficulty  citations,  a  typically 
high  volume  of  traffic  is  one  factor  that 
contributes  to  the  difficulty  of  some  sectors. 


/  -  ' 


92 


TABLE  4-6.  NUMBER  OF  CONTROLLERS  CITING  DIFFICULTY  FACTORS  IN  CHICAGO  SECTORS 


f 


Subtracting  the  traffic  volume'  citations  from 
the  total  number  of  citations  produces  a  rough 
measure  of  "residual"  difficulty  attributable  to 
factors  other  than  traffic. 

Because  the  measure  of  residual  difficulty 
excludes  cited  differences  in  traffic  volume, 
the  measure  appears  to  be  more  appropriate  than 
the  mean  difficulty  rating  for  use  as  a 
predictor  of  workloads  that  are  estimated  for  a 
fixed  level  of  traffic.  Nonetheless,  the 
residual  difficulty  measure  (based  on 
controllers'  unconstrained  statements)  shows  no 
practical  advantage  over  the  mean  difficulty 
rating  (based  on  controllers'  categorizations  on 
a  designated  scale)  in  the  present  data; 
similar  relationships  are  found  when  these 
measures  are  used  to  predict  workload  estimates 
for  different  sectors. 

4.5.2  A  Compar ison  Among  Workload  Estimators  Relative 
To  Control  Difficulty 

For  a  fixed  level  of  aircraft,  specifically  10 
aircraft  under  control,  workload-traffic 
functions  for  Rated  Wordload  (Table  4-4),  Judged 
Workload  (Table  4-5),  and  Computed  Workload 
(Figs.  4-3  and  4-4),  were  used  to  calculate  a 
workload  for  each  studied  sector.  These 
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workload  estimates  are  plotted  in  Fig.  4-6  as 
functions  of  the  mean  sector  difficulty,  and  in 
Fig.  4-7  as  functions  of  the  residual  sector 
difficulty  citations  (minus  those  for  traffic 
volume) . 

In  the  figures,  least  squares  lines  are  shown 
drawn  through  the  estimates  of  Computed  and 
Judged  Workload  for  the  sectors  in  each  control 
area. 

Figures  4-6  and  4-7  reveal  some  useful 
characteristics  of  the  Computed  Workload 
measure.  First,  the  measure  is  evidently  a 
sensitive  index  of  sector  control  difficulty; 
workload  is  higher,  the  greater  the  sector 
control  difficulty.  Second,  the  measure  is 
evidently  a  more  sensitive  index  of  sector 
control  difficulty  than  Judged  Workload; 
sectors  which  differ  in  terms  of  Computed 
Workload  do  not  all  differ  in  terms  of  Judged 
Workload.  Third,  the  measure  agrees  with  Rated 


Workload 

(workpace)  as 

ind icated 

by 

the 

consistent 

ordering  between 

sectors; 

in 

the 

Wing  area,  MLI  shows  less  Rated  and  less 
Computed  work  than  PIA,  and  in  the  Terminal 
area,  FARMM  shows  slightly  less  work  than  VAINS. 
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SUBJECT I VK  WORKLOAD  -  10  AIRCRAFT  COMPLIED  WORKLOAD  -  10  AIRCRAFT 

(MINUTES) 

FRY  VERY 

GUT  AYE RAG I  HEAVY 


The  impressive  consistency  between  Computed 
Workload  and  subjective  measures  that  indicate 
sector  differences  is  further  evidence 
suggesting  that  Computed  Workload  is  valid  from 
a  controller's  point  of  view.  Additional 
favorable  evidence  is  produced  by  the  next 
analysis,  which  concerns  the  sensitivity  of 
Computed  Workload  to  workload  variations  within 
an  individual  sector. 

4.6  CORRELATION  BETWEEN  CONCURRENT  MEASURES  OF  RATED  AND 
COMPUTED  WORKLOAD 

Workpace  ratings  were  made  months  before  Computed 
Workload  was  derived  offline  from  data  that  was 
recorded  automatically  while  the  rated  control 
activities  occurred.  Therefore,  in  a  real  sense,  we 
may  ask  whether  Rated  Workload  predicted  Computed 
Workload.  The  present  approach  to  this  question  is  to 
examine  the  degree  to  which  concurrent  values  of  Rated 
Workload  and  Computed  Workload  covary  in  a  mathematical 
sense . 

The  product  moment  correlations  between  concurrent 
values  of  the  two  variables  are  listed  in  Table  4-7. 
In  the  table,  correlations  are  given  for  Computed  Total 
workload,  as  well  as  each  of  the  three  workload 
components  comprising  the  total,  namely.  Routine, 
Surveillance,  and  Conflict  Prevention  Workload.  For 
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TABLE  4-7.  COMPUTED  WORKLOAD  PREDICTED  FROM  RATED  WORKLOAD 
(WORKPACE) 


SECTOR 

PREDICTED  VARIABLE 

r 

100xr2 

b 

a 

BDF 

ROUTINE 

0.69 

47.6 

1.39 

1.84 

(14) 

SURVEILLANCE 

0.67 

44.9 

0.57 

2.11 

CONFLICT  PREVENTION 

0.70 

49.0 

0.58 

0.26 

TOTAL 

0.75 

56.2 

2.55 

4.21 

MLI 

ROUTINE 

0.30 

9.0 

0.50 

2.36 

(54) 

SURVEILLANCE 

0.49 

24.0 

mSm 

1.47 

CONFLICT  PREVENTION 

0.53 

28.1 

■Hi 

TOTAL 

0.42 

17.6 

1.33 

4.10 

VAINS 

ROUTINE 

0.79 

62.4 

1.70 

■  -0.93 

(57) 

SURVEILLANCE 

0.89 

79.2 

0.68 

1.27 

CONFLICT  PREVENTION 

0.87 

75.7 

0.34 

-0.52 

TOTAL 

0.87 

75.7 

2.73 

-0.19 

FARMM 

ROUTINE 

0.78 

60.8 

1.26 

1.80 

(73) 

SURVEILLANCE 

0.78 

60.8 

0.75 

1.08 

CONFLICT  PREVENTION 

0.76 

57.8 

0.36 

-0.41 

TOTAL 

0.78 

60.8 

2.36 

2.46 

1 

ROUTINE 

0.78 

60.8 

1.56 

-1.15 

M 

SURVEILLANCE 

0.77 

59.3 

0.64 

0.17 

CONFLICT  PREVENTION 

0.74 

54.8 

0.82 

-1.60 

ii 

TOTAL 

0.83 

68.9 

3.01 

-2.57 

NOTE:  Each  computed  workload  variable  was  predicted  using  least  squares 
linear  regression;  N  =  24  for  each  fitted  function. 

r  =  product  moment  coefficient  of  correlation. 

100xr2  =  percentage  of  variance  accounted  for  assuming  a  linear  prediction 
rule. 

b  =  slope  of  least  squares  prediction  function, 
a  =  intercept  of  least  square  prediction  function. 


sectors  BDF,  VAINS,  FARMM,  and  PIA,  ratings  accounted 
for  an  impressive,  56-76  percent  of  the  Total  Workload 
variance.  For  MLI,  although  the  correlation  (r*0.42) 
is  reliably  greater  than  zero  (level  of  significance 
less  than  0.05)  as  in  the  other  sectors,  only  18 
percent  of  the  Computed  Workload  variance  is  accounted 
for.  We  cannot  ascertain  in  retrospect  which  of  the 
two  measures  erred  with  respect  to  the  workload  that 
the  R  controller  in  MLI  actually  experienced. 

The  operational  significance  of  these  correlations  is 
suggested  more  directly  by  Figs.  4-8  and  4-9,  which 
illustrate  the  way  that  the  two  workload  measures 
covary  within  each  hour  that  VAINS  (the  highest 
correlation)  and  MLI  (the  lowest  correlation)  were 
observed.  Clearly,  Computed  Workload  has  the 
capability  of  tracking  Rated  Workload  to  an  impressive 
degree . 

4.7  WORKLOAD  AT  CAPACITY 

Evidence  has  been  presented  suggesting  that  Computed 
Workload  is  both  a  sensitive  and  a  valid  measure  of 
controller  workload  in  today's  enroute  sectors. 
Because  the  empirical  workload-traffic  function  is 
essentially  linear  for  a  wide  range  of  traffic  levels, 
there  is  no  indication  in  the  function  of  an  upper 
limit.  As  a  result,  other  sources  of  information  must 
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FIGURE  4-8.  COVARIATION  OF  TOTAL  COMPUTED  WORKLOAD  AND 
RATED  WORKLOAD  IN  SECTOR  VAINS 


be  used  to  estimate  the  workload  associated  with 
traffic  capacity. 

Capacity  is  defined  as  the  maximum  number  of  aircraft 
that  the  average  R  controller  can  handle  safely  during 
an  interval  of  time  on  the  order  of  five  minutes.  This 
definition  is  consistent  with  the  conditions  that 
controllers  who  responded  to  the  questionnaire  on  the 
workload-traffic  relationship  (the  Appendix,  Interview 
3)  assumed.  The  controllers'  questionnaire  responses 
are  a  potential  source  of  the  additional  information 
that  is  needed. 

Workload  was  judged  on  the  workpace  scale,  which  has  an 
explicit  upper  limit,  "Very  Heavy,"  that  might  be  used 
to  estimate  the  capacity  or  near-capacity  traffic  level 
in  a  sector. 

To  perform  this  analysis,  the  linear  workload-traffic 
functions  for  questionnaire  judgments  (Table  4-5)  were 
used  to  estimate  two  traffic  levels,  one  associated 
with  "Average"  workload,  the  other  with  "Very  Heavy" 
workload.  The  traffic  estimates  are  shown  in  Table 
4-8.  An  analogous  procedure  was  used  to  derive  traffic 
levels  from  workpace  ratings  (Table  4-4);these  traffic 
levels  are  given  in  Table  4-9. 
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TABLE  4-8.  COMPUTED  WORKLOAD  FOR  SELECTED  WORKPACE  VALUES 
DERIVED  FROM  QUESTIONNAIRE  RESPONSES 


AREA 

AND 

SECTOR 

AIRCRAFT  FOR 
"AVERAGE" 
WORKLOAD 

COMPUTED 

WORKLOAD 

(MIN) 

AIRCRAFT  FOR 
"VERY  HEAVY" 
WORKLOAD 

COMPUTED 

WORKLOAD 

(MIN) 

WEST  WING 

MLI 

9.41 

9.82 

19.76 

22.90 

PIA 

9.81 

12.13 

18.14 

23.46 

WEST 

TERMINAL 

RFD 

8.07 

8.56 

15.21 

18.77 

WDPT 

6.57 

9.24 

14.68 

21.65 

VAINS 

6.13 

9.60 

15.81 

30.80 

FARMM 

6.66 

11.19 

13.98 

24.51 

WEST  HIGH 

MCK 

13.00 

13.44 

23.35 

25.55 

IOW 

11.07 

9.97 

21.07 

19.67 

JOT 

8.46 

12.50 

15.78 

21.06 

BDF 

9.28 

13.03 

17.61 

26.27 

DBQ 

10.53 

13.78 

19.35 

25.86 

MEAN 

11.21 

23.64 

NOTE:  Sectors  are  listed  in  each  area  by  increasing  judged  control  difficulty 


TABLE  4-9.  COMPUTED  WORKLOAD  FOR  SELECTED  WORKPACE  VALUES 
DERIVED  FROM  WORKPACE  RATINGS 


AREA 

AND 

SECTOR 

AIRCRAFT  FOR 
"AVERAGE" 
WORKLOAD 

COMPUTED 

WORKLOAD 

(MIN) 

AIRCRAFT  FOR 
"VERY  HEAVY" 
WORKLOAD 

COMPUTED 

WORKLOAD 

(MIN) 

WEST  WING 

ML  I 

13.13 

14.66 

24.94 

30.01 

PIA 

9.14 

11.22 

17.55 

22.66 

WEST 

TERMINAL 

VAINS 

6.74 

10.94 

11.54 

21.45 

farmm 

6.65 

11.17 

12.41 

21.66 

WEST  HIGH 

BDF 

11.54 

16.62 

19.60 

29.43 

MEAN 

12.92 

25.04 

NOTE:  Sectors  are  listed  In  each  area  by  increasing  judged  control  difficulty. 


There  are  apparently  no  published  data  to  compare  with 
the  traffic  estimated  for  "Very  Heavy"  workload.  But 
we  find  assumed  in  a  model  applied  to  O'Hare 
International  Airport  operations  that  an  approach 
controller's  traffic  workload  "may  never  exceed  14 
aircraft"  (Ref.  10,  p.  B-15)  under  control  at  one 
time.  Interestingly  enough,  the  four  sectors  we 
studied  in  the  West  Terminal  area,  which  interfaces 
with  O’Hare,  have  "Very  Heavy"  traffic  values  close  to 
14  aircraft,  specifically  13.98-15.81  aircraft.  Our 
application  of  the  "Very  Heavy"  traffic  estimates  to 
approximate  traffic  capacity  could,  therefore,  be 
val id . 

The  traffic  estimates  were  input  to  the 
workload-traffic  functions  for  Computed  Workload  (Figs. 
4-3  and  4-4),  in  order  to  calculate  the  amount  of  work 
for  the  two  work  levels.  These  workload  values  in 
minutes  are  shown  in  Tables  4-8  and  4-9  next  to  the 
input  traffic  levels  used  to  calculate  them. 

Ideally,  we  would  expect  to  see  in  Tables  4-8  and  4-9  a 
constant  value  of  Computed  Workload  associated  with 
"Average*  workpace,  and  a  higher  constant  value 
associated  with  "Very  Heavy"  workpace.  In  Table  4-8, 
we  see  that  the  workload  values  are  within  the  range 
8.56-13.78  minutes  for  "Average"  workpace,  and 
18.77-30.80  minutes  for  "Very  Heavy"  workpace. 
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Similarly,  in  Table  4-9,  the  calculated  workload  values 
are  11.17-16.62  for  "Average"  and  21.45-30.01  for  "Very 
Heavy"  workpaces,  respectively.  All  the  reasons  for 
this  variability,  which  is  partly  due  to  judgmental 
factors,  are  not  known,  but  constants  can  still  be 
derived  from  these  results  for  practical  purposes. 

To  estimate  a  constant  value  of  Computed  Workload  for 
future  analyses  that  require  a  cutoff  or  criterion,  as 
in  the  ATF  model,  we  shall  calculate  a  mean  workload 
using  the  workload  values  for  the  different  sectors. 
From  Table  4-8  (for  questionnaire  judgments) ,  we  obtain 
mean  workload  values  of  about  11  and  24  minutes  for 
"Average"  and  "Very  Heavy"  workpaces,  respectively. 
These  means  are  slightly  lower  than  the  values  one 
could  derive  from  Table  4-9  (for  workpace  ratings) , 
specifically,  13  and  25  minutes. 

These  estimated  values  of  workload  might  be  applied  in 
analyses  of  results  from  the  ATF  model,  or  in  analyses 
of  Computed  Workload  data.  In  such  cases,  the  “Very 
Heavy"  workload  estimate  would  be  used  as  an 
approximation  of  workload  at  capacity.  The  "Average" 
workload  estimate  could  be  used  to  detect  when  workload 
exceeds  the  sector  average  value. 
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4.8  CONVERGING  LINES  OP  EVIDENCE  REVIEWED 


This  analysis  has  presented  evidence  indicating: 

1.  The  workpace  ratings,  the  source  of  Rated  Workload 
estimates,  are  consistent  with  what  the  raters 
believed  are  typical  traffic  levels  in  the  studied 
sectors. 

2.  Computed  Workload  and  Rated  Workload  are  both  more 
strongly  related  to  Aircraft  Under  Control  than  to 
Traffic  Flow  Rate,  suggesting  they  reflect  the  same 
aspects  of  the  control  process. 

3.  Computed  Workload  and  Rated  Workload  are  both 

orderly  functions  of  judged  sector  control 

difficulty. 

4.  Computed  Workload  and  Rated  Workload  are 

substantially  correlated  with  each  other?  the 
former  can  track  the  latter  quite  closely  as  a 
function  of  time. 

From  these  results,  we  conclude  that  Computed  Workload 

is  a  sensitive,  valid  measure  of  R  controller  workload. 
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5. 


CONCLUSIONS 


5.1  VALIDATION  OF  WORKLOAD  ESTIMATES  MADE  BY  CONTROLLERS 

Two  independent  estimates  of  workload  made  by  journey¬ 
men  controllers  were  used  as  standards  in  evaluating 
RECEF  as  a  workload  model.  They  are  workpace  ratings 
made  in  real-time  for  specific  time  intervals  and 
long-term  judgments  of  workload  based  on  past  experi¬ 
ence  in  working  the  sectors.  The  two  estimates  are  in 
substantial  agreement  concerning  the  number  of  aircraft 
which  produce  various  workload  levels  in  the  five  sec¬ 
tors  for  which  workpace  ratings  were  taken.  It  is  con¬ 
cluded  that  the  two  forms  of  estimates  are  consistent 
within  themselves  and  therefore  serve  as  valid  stan¬ 
dards  for  evaluating  RECEP. 

5.2  VALIDATION  OF  RECEP  AS  A  WORKLOAD  INDEX 

Numerical  values  for  controller  workload  as  com outed  py 
RECEP  are  strongly  correlated  with  workpace  ratings 
taken  simultaneously  over  a  large  number  of  test  intei — 
vals.  In  four  of  the  five  rated  sectors,  workpace  rat¬ 
ings  accounted  for  56-16  percent  of  the  RECEP  workload 
variance.  Eighteen  percent  of  the  variance  was  ac¬ 
counted  for  In  the  fifth  sector,  MLI ,  where  traffic  was 
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significantly  oelow  average.  In  all  five  sectors,  the 
RECEP  workload  values  and  the  workoace  ratings  track 
each  other  as  a  function  of  time  to  an  impressive  de¬ 
gree.  These  findings  lead  to  the  conclusion  that  RECEP 
is  a  valid  index  of  the  level  of  workload  within  indi¬ 
vidual  sectors. 

Two  estimates  of  sector  relative  control  difficulty 
within  each  area  of  specia 11 za tion  were  ootaioed  from 
controller  interviews  and  questionnaires.  Workload  va¬ 
lues  associated  with  a  fixed  number  of  aircraft  under 
control  were  computed  from  three  sources*  RECEP,  work- 
pace  ratings,  and  judged  workload  from  questionnaire 
data.  The  workload  values  from  all  three  sources  were 
highly  correlated  with  the  two  estimates  of  relative 
sector  difficulty;  l.e.,  the  higher  the  sector  diffi¬ 
culty,  the  higher  the  workload.  RECEP  workload  appears 
to  oe  a  very  sensitive  indication  of  small  differences 
in  relative  sector  difficulty.  These  findings  lead  to 
the  conclusion  that  RECEP  values  are  a  valid  index  for 
comparing  workload  from  one  sector  to  another. 

5.3  VALIDATION  OF  RECEP  AS  A  WORKLOAD  PREDICTOR 

Workload  values  obtained  from  RECEP  computations,  work- 
pace  ratings  and  judged  workload  are  all  highly  corre- 
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lited  with  the  mean  numoer  of  aircraft  under  control 


(AUC).  AUC  accounts  for  greater  than  80  Dercent  of  the 
variance  in  RECEP  workload  in  all  eleven  sectors  stu¬ 
dies.  Correlation  coefficients  relatina  AUC  to  work- 
Dace  and  fudged  workload  range  from  0.49  to  0.89.  This 
rinding  is  a  s tronq  indication  that  the  three  measures 
of  workload  capture  the  same  aspects  of  the  control 
process.  It  also  leads  to  the  conclusion  that  RECEP  is 
a  valid  means  of  predicting  workload  within  lndividu'1 
sectors  for  specified  levels  of  traflic. 

8.4  CALIBRATION  OF  RECEP  AT  CAPACITY  WORKLOAD  CONDITIONS 

RECEP  workload  values  measured  over  a  large  number  of 
test  intervals  for  eleven  sectors  are  essentially  a  li¬ 
near  function  of  traffic  and.  therefore,  give  no  indi¬ 
cation  of  an  upper,  or  capacity,  limit.  RECEP  workload 
values  for  traffic  levels  associated  with  “very  heavy" 
workload  estimates  obtained  both  from  workoace  ratings 
and  from  judged  workload  were  investigated  as  a  possi¬ 
ble  upper  limit.  Unfortunately,  in  both  cases,  the  va¬ 
lues  varied  widely  from  sector  to  sector.  Thus,  it 
would  be  necessary  in  future  applications  of  RECEP/ATE 
to  calibrate  sectors  individually  for  capaci  ty  condi¬ 
tions.  This  is  an  unsatisfactory  solution.  A  mean 
value  for  all  sectors  which  corresponds  to  "very  heavy" 
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workload  appears  to  be  a  more  reasonable  choice.  Mean 
RECEP  workload  values  of  24  and  25  minutes  were  obta¬ 
ined  when  derived  from  "very  heavy"  judged  workload  and 
"very  heavy"  workpace  ratings,  respectively.  It  is  re¬ 
commended  that  24  minutes  be  used  to  represent  capacity 
workload,  recognizing  that  this  is  an  average  value  for 
many  sectors  and  controllers. 

It  is  interesting  to  note  that  RECEP  workload  values  at 
high  tralfic  levels  exceed  15  minutes  during  a 
15-minute  interval.  This  is  attributable  to  the  method 
used  in  computing  workload.  RECEP  estmates  the  time 
spent  on  individual  activities,  both  mental  and  physi¬ 
cal.  Many  of  these  activities  are  performed  simultane¬ 
ously!  thus,  it  is  possible  to  exceed  15  minutes  of 
work  in  15  minutes  of  elapsed  time.  RECEP  is  an  index 
of  controller  busyness  rather  than  an  aosolute  measure 
of  working  time  versus  idle  time. 
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APPENDIX:  CONTROLLER  SURVEY  TO  DETERMINE  SECTOR  WORKLOAD  LEVELS 


The  three  appended  questionnaire  forms  were  used  to  guide 
controlled  interviews  of  air  traffic  controllers  during  the 
Chicago  ARTCC  RECEP  validation  tests.  The  forms  were  designed  to 
provide  long-term  estimates  of  workload  levels  within  sectors 
which  were  used  to  compare  with  short-term  RECEP  measures  of  work¬ 
load  and  with  workpace  ratings.  Questionnaires  were  answered  in 
the  presence  of  an  interviewer  who  explained  the  meaning  of  each 
form  and  solicited  qualifying  information  when  it  appeared 
relevant.  Controllers  responded  for  only  those  sectors  in  which 
they  had  had  recent  R-position  experience.  A  two-controller  team 
(R  and  D  positions)  was  assumed.  The  three  forms  are  attached 
and  are  self-explanatory.  Forms  1  and  3  use  a  seven-point  rating 
scale  for  easy  comparison  with  workpace  ratings. 


DEPARTMENT  OF  TRANSPORTATION 

RESEARCH  AND  SPECIAL  PROGRAMS  ADMINISTRATION 

TRANSPORTATION  SYSTEMS  CENTER 
KENDALL  SQUARE,  CAMBRIDGE  MA  02142 

RELATIVE  SECTOR  CONTROL  DIFFICULTY 
Please  select  from  the  following  list  of  sectors  only  those  with  which 
you  have  worked  recently. 

AREA 

SECTOR 

List  these  sectors  below,  beside  the  vertical  scale  provided.  Arrange 
the  sectors  according  to  the  degree  of  RELATIVE  DIFFICULTY  you  believe 
typical  R  controllers  working  in  them  experience  during  typical  busy 
periods.  Assume  conditions  requiring  a  two-controller  (R  and  D)  team. 


Briefly  explain  to  the  extent  that  you  are  able  (you  are  of  course 
aware  that  the  cause  of  difficulty  is  not  always  easy  to  pinpoint) 
the  order  you  use. 


LEVEL  OF  RELATIVE  DIFFICULTY 


YOUR  LIST 


YOUR  EXPLANATION 


Most  Difficult  to  Control  E=> 
Very  Difficult 
Above-Average  Difficulty 
Average 

Below-Average  Difficulty  cSH 
Fairly  Easy 
Easiest  to  Control 


A  -2 


DEPARTMENT  OF  TRANSPORTATION 

RESEARCH  AND  SPECIAL  PROGRAMS  ADMINISTRATION 

TRANSPORTATION  SYSTEMS  CENTER 
KENDALL  SQUARE,  CAMBRIDGE.  MA  02142 


‘interview  2:  OVERALL  SECTOR  TRAFFIC  FLOW 


f Please  give  the  information  requested  below  for  sectors  with  which  you 


have  worked  recently.  Assume  typical  busy  conditions  requiring  a 


two-controller  (R  and  D)  team. 


SECTOR  FLIGHT  TIME 
Estimate  from  your  own 
experience  the  average 
time  in  minutes  an  air¬ 
craft  is  under  the 
sector's  control;  state 
<  any  qualifications  you 

wish. 


SECTOR  TRAFFIC  CAPACITY 
Estimate  from  your  own  ex¬ 
perience  the  maximum  number 
of  aircraft  that  might  be 
handed  off  to  the  sector 
during  one  hour,  without 
without  causing  a  typical 
R  controller  to  "go  under" 
or  "lose  the  picture;" 
state  any  qualifications 
you  wish. 


REA 


SECTOR 


DEPARTMENT  OF  TRANSPORTATION 

RESEARCH  AND  SPECIAL  PROGRAMS  ADMINISTRATION 

TRANSPORTATION  SYSTEMS  CENTER 
KENDALL  SQUARE,  CAMBRIDGE  MA  02142 


INTERVIEW  3:  SECTOR  WORKLOAD  GROWTH  RATES 


This  interview  asks  the  following  kind  of  question:  "When  20  aircraft 
are  controlled  at  the  same  time  by  sector  X,  what  degree  of  workload 
does  a  typical  R  controller  experience?"  This  kind  of  question  is 
asked  for  eight  levels  of  aircraft  traffic  in  each  of  several  sectors. 
The  degree  of  workload  you  decide  is  appropriate  is  selected  from  the 
following  list  of  seven  values: 


VH  =  Very 

Heavy 

H  = 

Heavy 

AA  =  Above 

Average 

A  = 

Average 

BA  =  Below 

Average 

L  = 

Light 

VL  =  Very 

Light 

TRAFFIC  LEVEL: 


An  example  of  the  workload  rating  procedure  is 
included  below  with  the  sectors  to  be  considered. 
Only  consider  sections  with  which  you  have  worked 
recently,  and  assume  conditions  requiring  a 
two-controller  (R  and  D)  team. 


NUMBER  OF  AIRCRAFT  UNDER  CONTROL  AT  ONE  TIME 


AREA 

SECTOR 

1-3 

4-6 

7-9 

10-12 

13-15 

16-18 

19-21 

22-24 

example 

X 

VL 

VL 

L 

A 

A 

H 

VH 

VH 
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